pyspark.sql.Column¶

class pyspark.sql.Column(jc: py4j.java_gateway.JavaObject)¶

A column in a DataFrame.

Column instances can be created by:

# 1. Select a column out of a DataFrame

df.colName
df["colName"]

# 2. Create from an expression
df.colName + 1
1 / df.colName

Methods

`alias`(alias, *kwargs)	Returns this column aliased with a new name or names (in the case of expressions that return more than one column, such as explode).
`asc`()	Returns a sort expression based on ascending order of the column.
`asc_nulls_first`()	Returns a sort expression based on ascending order of the column, and null values return before non-null values.
`asc_nulls_last`()	Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
`astype`(dataType)	`astype()` is an alias for `cast()`.
`between`(lowerBound, upperBound)	True if the current column is between the lower bound and upper bound, inclusive.
`bitwiseAND`(other)	Compute bitwise AND of this expression with another expression.
`bitwiseOR`(other)	Compute bitwise OR of this expression with another expression.
`bitwiseXOR`(other)	Compute bitwise XOR of this expression with another expression.
`cast`(dataType)	Casts the column into type `dataType`.
`contains`(other)	Contains the other element.
`desc`()	Returns a sort expression based on the descending order of the column.
`desc_nulls_first`()	Returns a sort expression based on the descending order of the column, and null values appear before non-null values.
`desc_nulls_last`()	Returns a sort expression based on the descending order of the column, and null values appear after non-null values.
`dropFields`(*fieldNames)	An expression that drops fields in `StructType` by name.
`endswith`(other)	String ends with.
`eqNullSafe`(other)	Equality test that is safe for null values.
`getField`(name)	An expression that gets a field by name in a `StructType`.
`getItem`(key)	An expression that gets an item at position `ordinal` out of a list, or gets an item by key out of a dict.
`ilike`(other)	SQL ILIKE expression (case insensitive LIKE).
`isNotNull`()	True if the current expression is NOT null.
`isNull`()	True if the current expression is null.
`isin`(*cols)	A boolean expression that is evaluated to true if the value of this expression is contained by the evaluated values of the arguments.
`like`(other)	SQL like expression.
`name`(alias, *kwargs)	`name()` is an alias for `alias()`.
`otherwise`(value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
`over`(window)	Define a windowing column.
`rlike`(other)	SQL RLIKE expression (LIKE with Regex).
`startswith`(other)	String starts with.
`substr`(startPos, length)	Return a `Column` which is a substring of the column.
`when`(condition, value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
`withField`(fieldName, col)	An expression that adds/replaces a field in `StructType` by name.

pyspark.sql.DataFrame

pyspark.sql.Observation