pyspark.streaming.StreamingContext.getOrCreate¶
-
classmethod
StreamingContext.getOrCreate(checkpointPath: str, setupFunc: Callable[], StreamingContext]) → pyspark.streaming.context.StreamingContext¶ Either recreate a StreamingContext from checkpoint data or create a new StreamingContext. If checkpoint data exists in the provided checkpointPath, then StreamingContext will be recreated from the checkpoint data. If the data does not exist, then the provided setupFunc will be used to create a new context.
- Parameters
- checkpointPathstr
Checkpoint directory used in an earlier streaming program
- setupFuncfunction
Function to create a new context and setup DStreams