Class GradientBoostingClassifierScikitsLearnNode

Gradient Boosting for classification.

This node has been automatically generated by wrapping the ``sklearn.ensemble.gradient_boosting.GradientBoostingClassifier`` class
from the ``sklearn`` library.  The wrapped instance can be accessed
through the ``scikits_alg`` attribute.

GB builds an additive model in a
forward stage-wise fashion; it allows for the optimization of
arbitrary differentiable loss functions. In each stage ``n_classes_``
regression trees are fit on the negative gradient of the
binomial or multinomial deviance loss function. Binary classification
is a special case where only a single regression tree is induced.

Read more in the :ref:`User Guide <gradient_boosting>`.

**Parameters**

loss : {'deviance', 'exponential'}, optional (default='deviance')
    loss function to be optimized. 'deviance' refers to
    deviance (= logistic regression) for classification
    with probabilistic outputs. For loss 'exponential' gradient
    boosting recovers the AdaBoost algorithm.

learning_rate : float, optional (default=0.1)
    learning rate shrinks the contribution of each tree by `learning_rate`.
    There is a trade-off between learning_rate and n_estimators.

n_estimators : int (default=100)
    The number of boosting stages to perform. Gradient boosting
    is fairly robust to over-fitting so a large number usually
    results in better performance.

max_depth : integer, optional (default=3)
    maximum depth of the individual regression estimators. The maximum
    depth limits the number of nodes in the tree. Tune this parameter
    for best performance; the best value depends on the interaction
    of the input variables.
    Ignored if ``max_leaf_nodes`` is not None.

min_samples_split : integer, optional (default=2)
    The minimum number of samples required to split an internal node.

min_samples_leaf : integer, optional (default=1)
    The minimum number of samples required to be at a leaf node.

min_weight_fraction_leaf : float, optional (default=0.)
    The minimum weighted fraction of the input samples required to be at a
    leaf node.

subsample : float, optional (default=1.0)
    The fraction of samples to be used for fitting the individual base
    learners. If smaller than 1.0 this results in Stochastic Gradient
    Boosting. `subsample` interacts with the parameter `n_estimators`.
    Choosing `subsample < 1.0` leads to a reduction of variance
    and an increase in bias.

max_features : int, float, string or None, optional (default=None)
    The number of features to consider when looking for the best split:

      - - If int, then consider `max_features` features at each split.
      - - If float, then `max_features` is a percentage and
      -   `int(max_features * n_features)` features are considered at each
      -   split.
      - - If "auto", then `max_features=sqrt(n_features)`.
      - - If "sqrt", then `max_features=sqrt(n_features)`.
      - - If "log2", then `max_features=log2(n_features)`.
      - - If None, then `max_features=n_features`.


    Choosing `max_features < n_features` leads to a reduction of variance
    and an increase in bias.

    Note: the search for a split does not stop until at least one
    valid partition of the node samples is found, even if it requires to
    effectively inspect more than ``max_features`` features.

max_leaf_nodes : int or None, optional (default=None)
    Grow trees with ``max_leaf_nodes`` in best-first fashion.
    Best nodes are defined as relative reduction in impurity.
    If None then unlimited number of leaf nodes.
    If not None then ``max_depth`` will be ignored.

init : BaseEstimator, None, optional (default=None)
    An estimator object that is used to compute the initial
    predictions. ``init`` has to provide ``fit`` and ``predict``.
    If None it uses ``loss.init_estimator``.

verbose : int, default: 0
    Enable verbose output. If 1 then it prints progress and performance
    once in a while (the more trees the lower the frequency). If greater
    than 1 then it prints progress and performance for every tree.

warm_start : bool, default: False
    When set to ``True``, reuse the solution of the previous call to fit
    and add more estimators to the ensemble, otherwise, just erase the
    previous solution.

random_state : int, RandomState instance or None, optional (default=None)
    If int, random_state is the seed used by the random number generator;
    If RandomState instance, random_state is the random number generator;
    If None, the random number generator is the RandomState instance used
    by `np.random`.

presort : bool or 'auto', optional (default='auto')
    Whether to presort the data to speed up the finding of best splits in
    fitting. Auto mode by default will use presorting on dense data and
    default to normal sorting on sparse data. Setting presort to true on
    sparse data will raise an error.

    .. versionadded:: 0.17
       *presort* parameter.

**Attributes**

``feature_importances_`` : array, shape = [n_features]
    The feature importances (the higher, the more important the feature).

``oob_improvement_`` : array, shape = [n_estimators]
    The improvement in loss (= deviance) on the out-of-bag samples
    relative to the previous iteration.
    ``oob_improvement_[0]`` is the improvement in
    loss of the first stage over the ``init`` estimator.

``train_score_`` : array, shape = [n_estimators]
    The i-th score ``train_score_[i]`` is the deviance (= loss) of the
    model at iteration ``i`` on the in-bag sample.
    If ``subsample == 1`` this is the deviance on the training data.

``loss_`` : LossFunction
    The concrete ``LossFunction`` object.

init : BaseEstimator
    The estimator that provides the initial predictions.
    Set via the ``init`` argument or ``loss.init_estimator``.

``estimators_`` : ndarray of DecisionTreeRegressor, shape = [n_estimators, ``loss_.K``]
    The collection of fitted sub-estimators. ``loss_.K`` is 1 for binary
    classification, otherwise n_classes.


See also

sklearn.tree.DecisionTreeClassifier, RandomForestClassifier
AdaBoostClassifier

**References**

J. Friedman, Greedy Function Approximation: A Gradient Boosting
Machine, The Annals of Statistics, Vol. 29, No. 5, 2001.

J. Friedman, Stochastic Gradient Boosting, 1999

T. Hastie, R. Tibshirani and J. Friedman.
Elements of Statistical Learning Ed. 2, Springer, 2009.

Overrides: object.__init__

Overrides: Node.is_invertible

(inherited documentation)

Overrides: Node.is_trainable

Predict class for X.

This node has been automatically generated by wrapping the sklearn.ensemble.gradient_boosting.GradientBoostingClassifier class from the sklearn library. The wrapped instance can be accessed through the scikits_alg attribute.

Parameters

X : array-like of shape = [n_samples, n_features]

The input samples.

Returns

y: array of shape = ["n_samples]

The predicted values.

Overrides: ClassifierNode.label

Fit the gradient boosting model.

This node has been automatically generated by wrapping the sklearn.ensemble.gradient_boosting.GradientBoostingClassifier class from the sklearn library. The wrapped instance can be accessed through the scikits_alg attribute.

Parameters

X : array-like, shape = [n_samples, n_features]

Training vectors, where n_samples is the number of samples and n_features is the number of features.

y : array-like, shape = [n_samples]

Target values (integers in classification, real numbers in regression) For classification, labels must correspond to classes.

sample_weight : array-like, shape = [n_samples] or None

Sample weights. If None, then samples are equally weighted. Splits that would create child nodes with net zero or negative weight are ignored while searching for a split in each node. In the case of classification, splits are also ignored if they would result in any single class carrying a negative weight in either child node.

monitor : callable, optional

The monitor is called after each iteration with the current iteration, a reference to the estimator and the local variables of _fit_stages as keyword arguments callable(i, self, locals()). If the callable returns True the fitting procedure is stopped. The monitor can be used for various things such as computing held-out estimates, early stopping, model introspect, and snapshoting.

Returns

self : object

Returns self.

Overrides: Node.stop_training