Package mdp :: Package nodes :: Class LogisticRegressionScikitsLearnNode
[hide private]
[frames] | no frames]

Class LogisticRegressionScikitsLearnNode



Logistic Regression (aka logit, MaxEnt) classifier.

This node has been automatically generated by wrapping the ``sklearn.linear_model.logistic.LogisticRegression`` class
from the ``sklearn`` library.  The wrapped instance can be accessed
through the ``scikits_alg`` attribute.

In the multiclass case, the training algorithm uses the one-vs-rest (OvR)
scheme if the 'multi_class' option is set to 'ovr' and uses the
cross-entropy loss, if the 'multi_class' option is set to 'multinomial'.
(Currently the 'multinomial' option is supported only by the 'lbfgs' and
'newton-cg' solvers.)

This class implements regularized logistic regression using the
`liblinear` library, newton-cg and lbfgs solvers. It can handle both
dense and sparse input. Use C-ordered arrays or CSR matrices containing
64-bit floats for optimal performance; any other input format will be
converted (and copied).

The newton-cg and lbfgs solvers support only L2 regularization with primal
formulation. The liblinear solver supports both L1 and L2 regularization,
with a dual formulation only for the L2 penalty.

Read more in the :ref:`User Guide <logistic_regression>`.

**Parameters**

penalty : str, 'l1' or 'l2'
    Used to specify the norm used in the penalization. The newton-cg and
    lbfgs solvers support only l2 penalties.

dual : bool
    Dual or primal formulation. Dual formulation is only implemented for
    l2 penalty with liblinear solver. Prefer dual=False when
    n_samples > n_features.

C : float, optional (default=1.0)
    Inverse of regularization strength; must be a positive float.
    Like in support vector machines, smaller values specify stronger
    regularization.

fit_intercept : bool, default: True
    Specifies if a constant (a.k.a. bias or intercept) should be
    added to the decision function.

intercept_scaling : float, default: 1
    Useful only if solver is liblinear.
    when self.fit_intercept is True, instance vector x becomes
    [x, self.intercept_scaling],
    i.e. a "synthetic" feature with constant value equals to
    intercept_scaling is appended to the instance vector.
    The intercept becomes intercept_scaling * synthetic feature weight
    Note! the synthetic feature weight is subject to l1/l2 regularization
    as all other features.
    To lessen the effect of regularization on synthetic feature weight
    (and therefore on the intercept) intercept_scaling has to be increased.

class_weight : dict or 'balanced', optional
    Weights associated with classes in the form ``{class_label: weight}``.
    If not given, all classes are supposed to have weight one.

    The "balanced" mode uses the values of y to automatically adjust
    weights inversely proportional to class frequencies in the input data
    as ``n_samples / (n_classes * np.bincount(y))``

    Note that these weights will be multiplied with sample_weight (passed
    through the fit method) if sample_weight is specified.

    .. versionadded:: 0.17
       *class_weight='balanced'* instead of deprecated *class_weight='auto'*.

max_iter : int
    Useful only for the newton-cg, sag and lbfgs solvers.
    Maximum number of iterations taken for the solvers to converge.

random_state : int seed, RandomState instance, or None (default)
    The seed of the pseudo random number generator to use when
    shuffling the data.

solver : {'newton-cg', 'lbfgs', 'liblinear', 'sag'}
    Algorithm to use in the optimization problem.

    - For small datasets, 'liblinear' is a good choice, whereas 'sag' is
        faster for large ones.
    - For multiclass problems, only 'newton-cg' and 'lbfgs' handle
        multinomial loss; 'sag' and 'liblinear' are limited to
        one-versus-rest schemes.
    - 'newton-cg', 'lbfgs' and 'sag' only handle L2 penalty.

    Note that 'sag' fast convergence is only guaranteed on features with
    approximately the same scale. You can preprocess the data with a
    scaler from sklearn.preprocessing.

    .. versionadded:: 0.17
       Stochastic Average Gradient descent solver.

tol : float, optional
    Tolerance for stopping criteria.

multi_class : str, {'ovr', 'multinomial'}
    Multiclass option can be either 'ovr' or 'multinomial'. If the option
    chosen is 'ovr', then a binary problem is fit for each label. Else
    the loss minimised is the multinomial loss fit across
    the entire probability distribution. Works only for the 'lbfgs'
    solver.

verbose : int
    For the liblinear and lbfgs solvers set verbose to any positive
    number for verbosity.

warm_start : bool, optional
    When set to True, reuse the solution of the previous call to fit as
    initialization, otherwise, just erase the previous solution.
    Useless for liblinear solver.

    .. versionadded:: 0.17
       *warm_start* to support *lbfgs*, *newton-cg*, *sag* solvers.

n_jobs : int, optional
    Number of CPU cores used during the cross-validation loop. If given
    a value of -1, all cores are used.

**Attributes**

``coef_`` : array, shape (n_classes, n_features)
    Coefficient of the features in the decision function.

``intercept_`` : array, shape (n_classes,)
    Intercept (a.k.a. bias) added to the decision function.
    If `fit_intercept` is set to False, the intercept is set to zero.

``n_iter_`` : array, shape (n_classes,) or (1, )
    Actual number of iterations for all classes. If binary or multinomial,
    it returns only 1 element. For liblinear solver, only the maximum
    number of iteration across all classes is given.

See also

SGDClassifier : incrementally trained logistic regression (when given
    the parameter ``loss="log"``).
sklearn.svm.LinearSVC : learns SVM models using the same algorithm.

**Notes**

The underlying C implementation uses a random number generator to
select features when fitting the model. It is thus not uncommon,
to have slightly different results for the same input data. If
that happens, try with a smaller tol parameter.

Predict output may not match that of standalone liblinear in certain
cases. See :ref:`differences from liblinear <liblinear_differences>`
in the narrative documentation.

**References**


LIBLINEAR -- A Library for Large Linear Classification
    http://www.csie.ntu.edu.tw/~cjlin/liblinear/

Hsiang-Fu Yu, Fang-Lan Huang, Chih-Jen Lin (2011). Dual coordinate descent
    methods for logistic regression and maximum entropy models.
    Machine Learning 85(1-2):41-75.
    http://www.csie.ntu.edu.tw/~cjlin/papers/maxent_dual.pdf

Instance Methods [hide private]
 
__init__(self, input_dim=None, output_dim=None, dtype=None, **kwargs)
Logistic Regression (aka logit, MaxEnt) classifier.
 
_get_supported_dtypes(self)
Return the list of dtypes supported by this node. The types can be specified in any format allowed by numpy.dtype.
 
_label(self, x)
 
_stop_training(self, **kwargs)
Transform the data and labels lists to array objects and reshape them.
 
label(self, x)
Predict class labels for samples in X.
 
stop_training(self, **kwargs)
Fit the model according to the given training data.

Inherited from PreserveDimNode (private): _set_input_dim, _set_output_dim

Inherited from unreachable.newobject: __long__, __native__, __nonzero__, __unicode__, next

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __subclasshook__

    Inherited from ClassifierCumulator
 
_check_train_args(self, x, labels)
 
_train(self, x, labels)
Cumulate all input data in a one dimensional list.
 
train(self, x, labels)
Cumulate all input data in a one dimensional list.
    Inherited from ClassifierNode
 
_execute(self, x)
 
_prob(self, x, *args, **kargs)
 
execute(self, x)
Process the data contained in x.
 
prob(self, x, *args, **kwargs)
Predict probability for each possible outcome.
 
rank(self, x, threshold=None)
Returns ordered list with all labels ordered according to prob(x) (e.g., [[3 1 2], [2 1 3], ...]).
    Inherited from Node
 
__add__(self, other)
 
__call__(self, x, *args, **kwargs)
Calling an instance of Node is equivalent to calling its execute method.
 
__repr__(self)
repr(x)
 
__str__(self)
str(x)
 
_check_input(self, x)
 
_check_output(self, y)
 
_get_train_seq(self)
 
_if_training_stop_training(self)
 
_inverse(self, x)
 
_pre_execution_checks(self, x)
This method contains all pre-execution checks.
 
_pre_inversion_checks(self, y)
This method contains all pre-inversion checks.
 
_refcast(self, x)
Helper function to cast arrays to the internal dtype.
 
_set_dtype(self, t)
 
copy(self, protocol=None)
Return a deep copy of the node.
 
get_current_train_phase(self)
Return the index of the current training phase.
 
get_dtype(self)
Return dtype.
 
get_input_dim(self)
Return input dimensions.
 
get_output_dim(self)
Return output dimensions.
 
get_remaining_train_phase(self)
Return the number of training phases still to accomplish.
 
get_supported_dtypes(self)
Return dtypes supported by the node as a list of dtype objects.
 
has_multiple_training_phases(self)
Return True if the node has multiple training phases.
 
inverse(self, y, *args, **kwargs)
Invert y.
 
is_training(self)
Return True if the node is in the training phase, False otherwise.
 
save(self, filename, protocol=-1)
Save a pickled serialization of the node to filename. If filename is None, return a string.
 
set_dtype(self, t)
Set internal structures' dtype.
 
set_input_dim(self, n)
Set input dimensions.
 
set_output_dim(self, n)
Set output dimensions.
Static Methods [hide private]
 
is_invertible()
Return True if the node can be inverted, False otherwise.
 
is_trainable()
Return True if the node can be trained, False otherwise.
Properties [hide private]

Inherited from object: __class__

    Inherited from Node
  _train_seq
List of tuples:
  dtype
dtype
  input_dim
Input dimensions
  output_dim
Output dimensions
  supported_dtypes
Supported dtypes
Method Details [hide private]

__init__(self, input_dim=None, output_dim=None, dtype=None, **kwargs)
(Constructor)

 

Logistic Regression (aka logit, MaxEnt) classifier.

This node has been automatically generated by wrapping the ``sklearn.linear_model.logistic.LogisticRegression`` class
from the ``sklearn`` library.  The wrapped instance can be accessed
through the ``scikits_alg`` attribute.

In the multiclass case, the training algorithm uses the one-vs-rest (OvR)
scheme if the 'multi_class' option is set to 'ovr' and uses the
cross-entropy loss, if the 'multi_class' option is set to 'multinomial'.
(Currently the 'multinomial' option is supported only by the 'lbfgs' and
'newton-cg' solvers.)

This class implements regularized logistic regression using the
`liblinear` library, newton-cg and lbfgs solvers. It can handle both
dense and sparse input. Use C-ordered arrays or CSR matrices containing
64-bit floats for optimal performance; any other input format will be
converted (and copied).

The newton-cg and lbfgs solvers support only L2 regularization with primal
formulation. The liblinear solver supports both L1 and L2 regularization,
with a dual formulation only for the L2 penalty.

Read more in the :ref:`User Guide <logistic_regression>`.

**Parameters**

penalty : str, 'l1' or 'l2'
    Used to specify the norm used in the penalization. The newton-cg and
    lbfgs solvers support only l2 penalties.

dual : bool
    Dual or primal formulation. Dual formulation is only implemented for
    l2 penalty with liblinear solver. Prefer dual=False when
    n_samples > n_features.

C : float, optional (default=1.0)
    Inverse of regularization strength; must be a positive float.
    Like in support vector machines, smaller values specify stronger
    regularization.

fit_intercept : bool, default: True
    Specifies if a constant (a.k.a. bias or intercept) should be
    added to the decision function.

intercept_scaling : float, default: 1
    Useful only if solver is liblinear.
    when self.fit_intercept is True, instance vector x becomes
    [x, self.intercept_scaling],
    i.e. a "synthetic" feature with constant value equals to
    intercept_scaling is appended to the instance vector.
    The intercept becomes intercept_scaling * synthetic feature weight
    Note! the synthetic feature weight is subject to l1/l2 regularization
    as all other features.
    To lessen the effect of regularization on synthetic feature weight
    (and therefore on the intercept) intercept_scaling has to be increased.

class_weight : dict or 'balanced', optional
    Weights associated with classes in the form ``{class_label: weight}``.
    If not given, all classes are supposed to have weight one.

    The "balanced" mode uses the values of y to automatically adjust
    weights inversely proportional to class frequencies in the input data
    as ``n_samples / (n_classes * np.bincount(y))``

    Note that these weights will be multiplied with sample_weight (passed
    through the fit method) if sample_weight is specified.

    .. versionadded:: 0.17
       *class_weight='balanced'* instead of deprecated *class_weight='auto'*.

max_iter : int
    Useful only for the newton-cg, sag and lbfgs solvers.
    Maximum number of iterations taken for the solvers to converge.

random_state : int seed, RandomState instance, or None (default)
    The seed of the pseudo random number generator to use when
    shuffling the data.

solver : {'newton-cg', 'lbfgs', 'liblinear', 'sag'}
    Algorithm to use in the optimization problem.

    - For small datasets, 'liblinear' is a good choice, whereas 'sag' is
        faster for large ones.
    - For multiclass problems, only 'newton-cg' and 'lbfgs' handle
        multinomial loss; 'sag' and 'liblinear' are limited to
        one-versus-rest schemes.
    - 'newton-cg', 'lbfgs' and 'sag' only handle L2 penalty.

    Note that 'sag' fast convergence is only guaranteed on features with
    approximately the same scale. You can preprocess the data with a
    scaler from sklearn.preprocessing.

    .. versionadded:: 0.17
       Stochastic Average Gradient descent solver.

tol : float, optional
    Tolerance for stopping criteria.

multi_class : str, {'ovr', 'multinomial'}
    Multiclass option can be either 'ovr' or 'multinomial'. If the option
    chosen is 'ovr', then a binary problem is fit for each label. Else
    the loss minimised is the multinomial loss fit across
    the entire probability distribution. Works only for the 'lbfgs'
    solver.

verbose : int
    For the liblinear and lbfgs solvers set verbose to any positive
    number for verbosity.

warm_start : bool, optional
    When set to True, reuse the solution of the previous call to fit as
    initialization, otherwise, just erase the previous solution.
    Useless for liblinear solver.

    .. versionadded:: 0.17
       *warm_start* to support *lbfgs*, *newton-cg*, *sag* solvers.

n_jobs : int, optional
    Number of CPU cores used during the cross-validation loop. If given
    a value of -1, all cores are used.

**Attributes**

``coef_`` : array, shape (n_classes, n_features)
    Coefficient of the features in the decision function.

``intercept_`` : array, shape (n_classes,)
    Intercept (a.k.a. bias) added to the decision function.
    If `fit_intercept` is set to False, the intercept is set to zero.

``n_iter_`` : array, shape (n_classes,) or (1, )
    Actual number of iterations for all classes. If binary or multinomial,
    it returns only 1 element. For liblinear solver, only the maximum
    number of iteration across all classes is given.

See also

SGDClassifier : incrementally trained logistic regression (when given
    the parameter ``loss="log"``).
sklearn.svm.LinearSVC : learns SVM models using the same algorithm.

**Notes**

The underlying C implementation uses a random number generator to
select features when fitting the model. It is thus not uncommon,
to have slightly different results for the same input data. If
that happens, try with a smaller tol parameter.

Predict output may not match that of standalone liblinear in certain
cases. See :ref:`differences from liblinear <liblinear_differences>`
in the narrative documentation.

**References**


LIBLINEAR -- A Library for Large Linear Classification
    http://www.csie.ntu.edu.tw/~cjlin/liblinear/

Hsiang-Fu Yu, Fang-Lan Huang, Chih-Jen Lin (2011). Dual coordinate descent
    methods for logistic regression and maximum entropy models.
    Machine Learning 85(1-2):41-75.
    http://www.csie.ntu.edu.tw/~cjlin/papers/maxent_dual.pdf

Overrides: object.__init__

_get_supported_dtypes(self)

 
Return the list of dtypes supported by this node. The types can be specified in any format allowed by numpy.dtype.
Overrides: Node._get_supported_dtypes

_label(self, x)

 
Overrides: ClassifierNode._label

_stop_training(self, **kwargs)

 
Transform the data and labels lists to array objects and reshape them.

Overrides: Node._stop_training

is_invertible()
Static Method

 
Return True if the node can be inverted, False otherwise.
Overrides: Node.is_invertible
(inherited documentation)

is_trainable()
Static Method

 
Return True if the node can be trained, False otherwise.
Overrides: Node.is_trainable

label(self, x)

 

Predict class labels for samples in X.

This node has been automatically generated by wrapping the sklearn.linear_model.logistic.LogisticRegression class from the sklearn library. The wrapped instance can be accessed through the scikits_alg attribute.

Parameters

X : {array-like, sparse matrix}, shape = [n_samples, n_features]
Samples.

Returns

C : array, shape = [n_samples]
Predicted class label per sample.
Overrides: ClassifierNode.label

stop_training(self, **kwargs)

 

Fit the model according to the given training data.

This node has been automatically generated by wrapping the ``sklearn.linear_model.logistic.LogisticRegression`` class
from the ``sklearn`` library.  The wrapped instance can be accessed
through the ``scikits_alg`` attribute.

**Parameters**

X : {array-like, sparse matrix}, shape (n_samples, n_features)
    Training vector, where n_samples in the number of samples and
    n_features is the number of features.

y : array-like, shape (n_samples,)
    Target vector relative to X.

sample_weight : array-like, shape (n_samples,) optional
    Array of weights that are assigned to individual samples.
    If not provided, then each sample is given unit weight.

    .. versionadded:: 0.17
       *sample_weight* support to LogisticRegression.

Returns

self : object
    Returns self.

Overrides: Node.stop_training