pyspark.pandas.DatetimeIndex¶

class pyspark.pandas.DatetimeIndex¶

Immutable ndarray-like of datetime64 data.

Parameters

dataarray-like (1-dimensional), optional

Optional datetime-like data to construct index with.

freqstr or pandas offset object, optional

One of pandas date offset strings or corresponding objects. The string ‘infer’ can be passed in order to set the frequency of the index as the inferred frequency upon creation.

normalizebool, default False

Normalize start/end dates to midnight before generating date range.

closed{‘left’, ‘right’}, optional

Set whether to include start and end that are on the boundary. The default includes boundary points on either end.

ambiguous‘infer’, bool-ndarray, ‘NaT’, default ‘raise’

When clocks moved backward due to DST, ambiguous times may arise. For example in Central European Time (UTC+01), when going from 03:00 DST to 02:00 non-DST, 02:30:00 local time occurs both at 00:30:00 UTC and at 01:30:00 UTC. In such a situation, the ambiguous parameter dictates how ambiguous times should be handled.

‘infer’ will attempt to infer fall dst-transition hours based on order
bool-ndarray where True signifies a DST time, False signifies a non-DST time (note that this flag is only applicable for ambiguous times)
‘NaT’ will return NaT where there are ambiguous times
‘raise’ will raise an AmbiguousTimeError if there are ambiguous times.

dayfirstbool, default False

If True, parse dates in data with the day first order.

yearfirstbool, default False

If True parse dates in data with the year first order.

dtypenumpy.dtype or str, default None

Note that the only NumPy dtype allowed is ‘datetime64[ns]’.

copybool, default False

Make a copy of input ndarray.

namelabel, default None

Name to be stored in the index.

See also

Index: The base pandas Index type.
to_datetime: Convert argument to datetime.

Examples

>>> ps.DatetimeIndex(['1970-01-01', '1970-01-01', '1970-01-01'])
DatetimeIndex(['1970-01-01', '1970-01-01', '1970-01-01'], dtype='datetime64[ns]', freq=None)

From a Series:

>>> from datetime import datetime
>>> s = ps.Series([datetime(2021, 3, 1), datetime(2021, 3, 2)], index=[10, 20])
>>> ps.DatetimeIndex(s)
DatetimeIndex(['2021-03-01', '2021-03-02'], dtype='datetime64[ns]', freq=None)

From an Index:

>>> idx = ps.DatetimeIndex(['1970-01-01', '1970-01-01', '1970-01-01'])
>>> ps.DatetimeIndex(idx)
DatetimeIndex(['1970-01-01', '1970-01-01', '1970-01-01'], dtype='datetime64[ns]', freq=None)

Methods

`all`(args, *kwargs)	Return whether all elements are True.
`any`([axis])	Return whether any element is True.
`append`(other)	Append a collection of Index options together.
`argmax`()	Return a maximum argument indexer.
`argmin`()	Return a minimum argument indexer.
`asof`(label)	Return the label from the index, or, if not present, the previous one.
`astype`(dtype)	Cast a pandas-on-Spark object to a specified dtype `dtype`.
`ceil`(freq, args, *kwargs)	Perform ceil operation on the data to the specified freq.
`copy`([name, deep])	Make a copy of this object.
`day_name`([locale])	Return the day names of the series with specified locale.
`delete`(loc)	Make new Index with passed location(-s) deleted.
`difference`(other[, sort])	Return a new Index with elements from the index that are not in other.
`drop`(labels)	Make new Index with passed list of labels deleted.
`drop_duplicates`([keep])	Return Index with duplicate values removed.
`droplevel`(level)	Return index with requested level(s) removed.
`dropna`([how])	Return Index or MultiIndex without NA/NaN values
`equals`(other)	Determine if two Index objects contain the same elements.
`factorize`([sort, na_sentinel])	Encode the object as an enumerated type or categorical variable.
`fillna`(value)	Fill NA/NaN values with the specified value.
`floor`(freq, args, *kwargs)	Perform floor operation on the data to the specified freq.
`get_level_values`(level)	Return Index if a valid level is given.
`holds_integer`()	Whether the type is an integer type.
`identical`(other)	Similar to equals, but check that other comparable attributes are also equal.
`indexer_at_time`(time[, asof])	Return index locations of values at particular time of day (example: 9:30AM).
`indexer_between_time`(start_time, end_time[, …])	Return index locations of values between particular times of day (example: 9:00-9:30AM).
`insert`(loc, item)	Make new Index inserting new item at location.
`intersection`(other)	Form the intersection of two Index objects.
`is_boolean`()	Return if the current index type is a boolean type.
`is_categorical`()	Return if the current index type is a categorical type.
`is_floating`()	Return if the current index type is a floating type.
`is_integer`()	Return if the current index type is a integer type.
`is_interval`()	Return if the current index type is an interval type.
`is_numeric`()	Return if the current index type is a numeric type.
`is_object`()	Return if the current index type is a object type.
`is_type_compatible`(kind)	Whether the index type is compatible with the provided type.
`isin`(values)	Check whether values are contained in Series or Index.
`isna`()	Detect existing (non-missing) values.
`isnull`()	Detect existing (non-missing) values.
`item`()	Return the first element of the underlying data as a python scalar.
`map`(mapper[, na_action])	Map values using input correspondence (a dict, Series, or function).
`max`()	Return the maximum value of the Index.
`min`()	Return the minimum value of the Index.
`month_name`([locale])	Return the month names of the DatetimeIndex with specified locale.
`normalize`()	Convert times to midnight.
`notna`()	Detect existing (non-missing) values.
`notnull`()	Detect existing (non-missing) values.
`nunique`([dropna, approx, rsd])	Return number of unique elements in the object.
`rename`(name[, inplace])	Alter Index or MultiIndex name.
`repeat`(repeats)	Repeat elements of a Index/MultiIndex.
`round`(freq, args, *kwargs)	Perform round operation on the data to the specified freq.
`set_names`(names[, level, inplace])	Set Index or MultiIndex name.
`shift`([periods, fill_value])	Shift Series/Index by desired number of periods.
`sort`(args, *kwargs)	Use sort_values instead.
`sort_values`([return_indexer, ascending])	Return a sorted copy of the index, and optionally return the indices that sorted the index itself.
`strftime`(date_format)	Convert to a string Index using specified date_format.
`symmetric_difference`(other[, result_name, sort])	Compute the symmetric difference of two Index objects.
`take`(indices)	Return the elements in the given positional indices along an axis.
`to_frame`([index, name])	Create a DataFrame with a column containing the Index.
`to_list`()	Return a list of the values.
`to_numpy`([dtype, copy])	A NumPy ndarray representing the values in this Index or MultiIndex.
`to_pandas`()	Return a pandas Index.
`to_series`([name])	Create a Series with both index and values equal to the index keys useful with map for returning an indexer based on an index.
`tolist`()	Return a list of the values.
`transpose`()	Return the transpose, For index, It will be index itself.
`union`(other[, sort])	Form the union of two Index objects.
`unique`([level])	Return unique values in the index.
`value_counts`([normalize, sort, ascending, …])	Return a Series containing counts of unique values.
`view`()	this is defined as a copy with the same identity

Attributes

`T`	Return the transpose, For index, It will be index itself.
`asi8`	Integer representation of the values.
`day`	The days of the datetime.
`day_of_week`	The day of the week with Monday=0, Sunday=6.
`day_of_year`	The ordinal day of the year.
`dayofweek`	The day of the week with Monday=0, Sunday=6.
`dayofyear`	The ordinal day of the year.
`days_in_month`	The number of days in the month.
`daysinmonth`	The number of days in the month.
`dtype`	Return the dtype object of the underlying data.
`empty`	Returns true if the current object is empty.
`has_duplicates`	If index has duplicates, return True, otherwise False.
`hasnans`	Return True if it has any missing values.
`hour`	The hours of the datetime.
`inferred_type`	Return a string of the type inferred from the values.
`is_all_dates`	Return if all data types of the index are datetime.
`is_leap_year`	Boolean indicator if the date belongs to a leap year.
`is_monotonic`	Return boolean if values in the object are monotonically increasing.
`is_monotonic_decreasing`	Return boolean if values in the object are monotonically decreasing.
`is_monotonic_increasing`	Return boolean if values in the object are monotonically increasing.
`is_month_end`	Indicates whether the date is the last day of the month.
`is_month_start`	Indicates whether the date is the first day of the month.
`is_quarter_end`	Indicator for whether the date is the last day of a quarter.
`is_quarter_start`	Indicator for whether the date is the first day of a quarter.
`is_unique`	Return if the index has unique values.
`is_year_end`	Indicate whether the date is the last day of the year.
`is_year_start`	Indicate whether the date is the first day of a year.
`microsecond`	The microseconds of the datetime.
`minute`	The minutes of the datetime.
`month`	The month of the timestamp as January = 1 December = 12.
`name`	Return name of the Index.
`names`	Return names of the Index.
`ndim`	Return an int representing the number of array dimensions.
`nlevels`	Number of levels in Index & MultiIndex.
`quarter`	The quarter of the date.
`second`	The seconds of the datetime.
`shape`	Return a tuple of the shape of the underlying data.
`size`	Return an int representing the number of elements in this object.
`values`	Return an array representing the data in the Index.
`week`	The week ordinal of the year.
`weekday`	The day of the week with Monday=0, Sunday=6.
`weekofyear`	The week ordinal of the year.
`year`	The year of the datetime.

pyspark.pandas.MultiIndex.sort_values

pyspark.pandas.DatetimeIndex.year