LogisticRegressionModel¶

class pyspark.mllib.classification.LogisticRegressionModel(weights: pyspark.mllib.linalg.Vector, intercept: float, numFeatures: int, numClasses: int)¶

Classification model trained using Multinomial/Binary Logistic Regression.

Parameters

weightspyspark.mllib.linalg.Vector: Weights computed for every feature.
interceptfloat: Intercept computed for this model. (Only used in Binary Logistic Regression. In Multinomial Logistic Regression, the intercepts will not be a single value, so the intercepts will be part of the weights.)
numFeaturesint: The dimension of the features.
numClassesint: The number of possible outcomes for k classes classification problem in Multinomial Logistic Regression. By default, it is binary logistic regression so numClasses will be set to 2.

Examples

>>> from pyspark.mllib.linalg import SparseVector
>>> data = [
...     LabeledPoint(0.0, [0.0, 1.0]),
...     LabeledPoint(1.0, [1.0, 0.0]),
... ]
>>> lrm = LogisticRegressionWithSGD.train(sc.parallelize(data), iterations=10)
>>> lrm.predict([1.0, 0.0])
1
>>> lrm.predict([0.0, 1.0])
0
>>> lrm.predict(sc.parallelize([[1.0, 0.0], [0.0, 1.0]])).collect()
[1, 0]
>>> lrm.clearThreshold()
>>> lrm.predict([0.0, 1.0])
0.279...

>>> sparse_data = [
...     LabeledPoint(0.0, SparseVector(2, {0: 0.0})),
...     LabeledPoint(1.0, SparseVector(2, {1: 1.0})),
...     LabeledPoint(0.0, SparseVector(2, {0: 1.0})),
...     LabeledPoint(1.0, SparseVector(2, {1: 2.0}))
... ]
>>> lrm = LogisticRegressionWithSGD.train(sc.parallelize(sparse_data), iterations=10)
>>> lrm.predict(numpy.array([0.0, 1.0]))
1
>>> lrm.predict(numpy.array([1.0, 0.0]))
0
>>> lrm.predict(SparseVector(2, {1: 1.0}))
1
>>> lrm.predict(SparseVector(2, {0: 1.0}))
0
>>> import os, tempfile
>>> path = tempfile.mkdtemp()
>>> lrm.save(sc, path)
>>> sameModel = LogisticRegressionModel.load(sc, path)
>>> sameModel.predict(numpy.array([0.0, 1.0]))
1
>>> sameModel.predict(SparseVector(2, {0: 1.0}))
0
>>> from shutil import rmtree
>>> try:
...    rmtree(path)
... except BaseException:
...    pass
>>> multi_class_data = [
...     LabeledPoint(0.0, [0.0, 1.0, 0.0]),
...     LabeledPoint(1.0, [1.0, 0.0, 0.0]),
...     LabeledPoint(2.0, [0.0, 0.0, 1.0])
... ]
>>> data = sc.parallelize(multi_class_data)
>>> mcm = LogisticRegressionWithLBFGS.train(data, iterations=10, numClasses=3)
>>> mcm.predict([0.0, 0.5, 0.0])
0
>>> mcm.predict([0.8, 0.0, 0.0])
1
>>> mcm.predict([0.0, 0.0, 0.3])
2

Methods

`clearThreshold`()	Clears the threshold so that predict will output raw prediction scores.
`load`(sc, path)	Load a model from the given path.
`predict`(x)	Predict values for a single data point or an RDD of points using the model trained.
`save`(sc, path)	Save this model to the given path.
`setThreshold`(value)	Sets the threshold that separates positive predictions from negative predictions.

Attributes

`intercept`	Intercept computed for this model.
`numClasses`	Number of possible outcomes for k classes classification problem in Multinomial Logistic Regression.
`numFeatures`	Dimension of the features.
`threshold`	Returns the threshold (if any) used for converting raw prediction scores into 0/1 predictions.
`weights`	Weights computed for every feature.

Methods Documentation

clearThreshold() → None¶: Clears the threshold so that predict will output raw prediction scores. It is used for binary classification only.

classmethod load(sc: pyspark.context.SparkContext, path: str) → pyspark.mllib.classification.LogisticRegressionModel ¶: Load a model from the given path.

predict(x: Union[VectorLike, pyspark.rdd.RDD[VectorLike]]) → Union[pyspark.rdd.RDD[Union[int, float]], int, float]¶: Predict values for a single data point or an RDD of points using the model trained.

save(sc: pyspark.context.SparkContext, path: str) → None¶: Save this model to the given path.

setThreshold(value: float) → None¶: Sets the threshold that separates positive predictions from negative predictions. An example with prediction score greater than or equal to this threshold is identified as a positive, and negative otherwise. It is used for binary classification only.

Attributes Documentation

intercept¶: Intercept computed for this model.

numClasses¶: Number of possible outcomes for k classes classification problem in Multinomial Logistic Regression.

numFeatures¶: Dimension of the features.

threshold¶: Returns the threshold (if any) used for converting raw prediction scores into 0/1 predictions. It is used for binary classification only.

weights¶: Weights computed for every feature.

MLlib (RDD-based)

LogisticRegressionWithSGD