pyspark.sql.functions.arrays_overlap¶
-
pyspark.sql.functions.arrays_overlap(a1: ColumnOrName, a2: ColumnOrName) → pyspark.sql.column.Column¶ Collection function: returns true if the arrays contain any common non-null element; if not, returns null if both the arrays are non-empty and any of them contains a null element; returns false otherwise.
Examples
>>> df = spark.createDataFrame([(["a", "b"], ["b", "c"]), (["a"], ["b", "c"])], ['x', 'y']) >>> df.select(arrays_overlap(df.x, df.y).alias("overlap")).collect() [Row(overlap=True), Row(overlap=False)]