package java
Spark streaming's Java API.
- Alphabetic
- By Inheritance
- java
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
class
JavaDStream[T] extends AbstractJavaDStreamLike[T, JavaDStream[T], JavaRDD[T]]
A Java-friendly interface to org.apache.spark.streaming.dstream.DStream, the basic abstraction in Spark Streaming that represents a continuous stream of data.
A Java-friendly interface to org.apache.spark.streaming.dstream.DStream, the basic abstraction in Spark Streaming that represents a continuous stream of data. DStreams can either be created from live data (such as, data from TCP sockets, Kafka, etc.) or it can be generated by transforming existing DStreams using operations such as
map
,window
. For operations applicable to key-value pair DStreams, see org.apache.spark.streaming.api.java.JavaPairDStream. - trait JavaDStreamLike[T, This <: JavaDStreamLike[T, This, R], R <: JavaRDDLike[T, R]] extends Serializable
-
class
JavaInputDStream[T] extends JavaDStream[T]
A Java-friendly interface to org.apache.spark.streaming.dstream.InputDStream.
-
class
JavaMapWithStateDStream[KeyType, ValueType, StateType, MappedType] extends JavaDStream[MappedType]
DStream representing the stream of data generated by
mapWithState
operation on a JavaPairDStream.DStream representing the stream of data generated by
mapWithState
operation on a JavaPairDStream. Additionally, it also gives access to the stream of state snapshots, that is, the state data of all keys after a batch has updated them.- KeyType
Class of the keys
- ValueType
Class of the values
- StateType
Class of the state data
- MappedType
Class of the mapped data
-
class
JavaPairDStream[K, V] extends AbstractJavaDStreamLike[(K, V), JavaPairDStream[K, V], JavaPairRDD[K, V]]
A Java-friendly interface to a DStream of key-value pairs, which provides extra methods like
reduceByKey
andjoin
. -
class
JavaPairInputDStream[K, V] extends JavaPairDStream[K, V]
A Java-friendly interface to org.apache.spark.streaming.dstream.InputDStream of key-value pairs.
-
class
JavaPairReceiverInputDStream[K, V] extends JavaPairInputDStream[K, V]
A Java-friendly interface to org.apache.spark.streaming.dstream.ReceiverInputDStream, the abstract class for defining any input stream that receives data over the network.
-
class
JavaReceiverInputDStream[T] extends JavaInputDStream[T]
A Java-friendly interface to org.apache.spark.streaming.dstream.ReceiverInputDStream, the abstract class for defining any input stream that receives data over the network.
-
class
JavaStreamingContext extends Closeable
A Java-friendly version of org.apache.spark.streaming.StreamingContext which is the main entry point for Spark Streaming functionality.
A Java-friendly version of org.apache.spark.streaming.StreamingContext which is the main entry point for Spark Streaming functionality. It provides methods to create org.apache.spark.streaming.api.java.JavaDStream and org.apache.spark.streaming.api.java.JavaPairDStream from input sources. The internal org.apache.spark.api.java.JavaSparkContext (see core Spark documentation) can be accessed using
context.sparkContext
. After creating and transforming DStreams, the streaming computation can be started and stopped usingcontext.start()
andcontext.stop()
, respectively.context.awaitTermination()
allows the current thread to wait for the termination of a context bystop()
or by an exception.
Value Members
- object JavaDStream extends Serializable
- object JavaInputDStream extends Serializable
- object JavaPairDStream extends Serializable
- object JavaPairInputDStream extends Serializable
- object JavaPairReceiverInputDStream extends Serializable
- object JavaReceiverInputDStream extends Serializable
-
object
JavaStreamingContext
JavaStreamingContext object contains a number of utility functions.