pyspark.sql.functions.array_remove¶

pyspark.sql.functions.array_remove(col: ColumnOrName, element: Any) → pyspark.sql.column.Column¶

Collection function: Remove all elements that equal to element from the given array.

Parameters

colColumn or str: name of column containing array
element :: element to be removed from the array

Examples

>>> df = spark.createDataFrame([([1, 2, 3, 1, 1],), ([],)], ['data'])
>>> df.select(array_remove(df.data, 1)).collect()
[Row(array_remove(data, 1)=[2, 3]), Row(array_remove(data, 1)=[])]

pyspark.sql.functions.array_sort

pyspark.sql.functions.array_distinct