pyspark.pandas.notna

pyspark.pandas.notna(obj)

Detect existing (non-missing) values.

Return a boolean same-sized object indicating if the values are not NA. Non-missing values get mapped to True. NA values, such as None or numpy.NaN, get mapped to False values.

Returns
bool or array-like of bool

Mask of bool values for each element that indicates whether an element is not an NA value.

See also

isna

Detect missing values for an array-like object.

Series.notna

Boolean inverse of Series.isna.

DataFrame.notnull

Boolean inverse of DataFrame.isnull.

Index.notna

Boolean inverse of Index.isna.

Index.notnull

Boolean inverse of Index.isnull.

Examples

Show which entries in a DataFrame are not NA.

>>> df = ps.DataFrame({'age': [5, 6, np.NaN],
...                    'born': [pd.NaT, pd.Timestamp('1939-05-27'),
...                             pd.Timestamp('1940-04-25')],
...                    'name': ['Alfred', 'Batman', ''],
...                    'toy': [None, 'Batmobile', 'Joker']})
>>> df
   age       born    name        toy
0  5.0        NaT  Alfred       None
1  6.0 1939-05-27  Batman  Batmobile
2  NaN 1940-04-25              Joker
>>> df.notnull()
     age   born  name    toy
0   True  False  True  False
1   True   True  True   True
2  False   True  True   True

Show which entries in a Series are not NA.

>>> ser = ps.Series([5, 6, np.NaN])
>>> ser
0    5.0
1    6.0
2    NaN
dtype: float64
>>> ps.notna(ser)
0     True
1     True
2    False
dtype: bool
>>> ps.notna(ser.index)
True