Packages

package io

IO codecs used for compression. See org.apache.spark.io.CompressionCodec.

Linear Supertypes
AnyRef, Any

Type Members

  1. trait CompressionCodec extends AnyRef

    :: DeveloperApi :: CompressionCodec allows the customization of choosing different compression implementations to be used in block storage.

    :: DeveloperApi :: CompressionCodec allows the customization of choosing different compression implementations to be used in block storage.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for a codec is not guaranteed compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  2. class LZ4CompressionCodec extends CompressionCodec

    :: DeveloperApi :: LZ4 implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: LZ4 implementation of org.apache.spark.io.CompressionCodec. Block size can be configured by spark.io.compression.lz4.blockSize.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  3. class LZFCompressionCodec extends CompressionCodec

    :: DeveloperApi :: LZF implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: LZF implementation of org.apache.spark.io.CompressionCodec.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  4. final class NioBufferedFileInputStream extends InputStream
  5. class ReadAheadInputStream extends InputStream
  6. class SnappyCompressionCodec extends CompressionCodec

    :: DeveloperApi :: Snappy implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: Snappy implementation of org.apache.spark.io.CompressionCodec. Block size can be configured by spark.io.compression.snappy.blockSize.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  7. class ZStdCompressionCodec extends CompressionCodec

    :: DeveloperApi :: ZStandard implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: ZStandard implementation of org.apache.spark.io.CompressionCodec. For more details see - http://facebook.github.io/zstd/

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

Inherited from AnyRef

Inherited from Any

Ungrouped