
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/release_highlights/plot_release_highlights_0_23_0.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_0_23_0.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_release_highlights_plot_release_highlights_0_23_0.py:


========================================
Release Highlights for scikit-learn 0.23
========================================

.. currentmodule:: sklearn

We are pleased to announce the release of scikit-learn 0.23! Many bug fixes
and improvements were added, as well as some new key features. We detail
below a few of the major features of this release. **For an exhaustive list of
all the changes**, please refer to the :ref:`release notes <changes_0_23>`.

To install the latest version (with pip)::

    pip install --upgrade scikit-learn

or with conda::

    conda install scikit-learn

.. GENERATED FROM PYTHON SOURCE LINES 24-35

Generalized Linear Models, and Poisson loss for gradient boosting
-----------------------------------------------------------------
Long-awaited Generalized Linear Models with non-normal loss functions are now
available. In particular, three new regressors were implemented:
:class:`~sklearn.linear_model.PoissonRegressor`,
:class:`~sklearn.linear_model.GammaRegressor`, and
:class:`~sklearn.linear_model.TweedieRegressor`. The Poisson regressor can be
used to model positive integer counts, or relative frequencies. Read more in
the :ref:`User Guide <Generalized_linear_regression>`. Additionally,
:class:`~sklearn.ensemble.HistGradientBoostingRegressor` supports a new
'poisson' loss as well.

.. GENERATED FROM PYTHON SOURCE LINES 35-55

.. code-block:: default


    import numpy as np
    from sklearn.model_selection import train_test_split
    from sklearn.linear_model import PoissonRegressor
    from sklearn.experimental import enable_hist_gradient_boosting  # noqa
    from sklearn.ensemble import HistGradientBoostingRegressor

    n_samples, n_features = 1000, 20
    rng = np.random.RandomState(0)
    X = rng.randn(n_samples, n_features)
    # positive integer target correlated with X[:, 5] with many zeros:
    y = rng.poisson(lam=np.exp(X[:, 5]) / 2)
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=rng)
    glm = PoissonRegressor()
    gbdt = HistGradientBoostingRegressor(loss='poisson', learning_rate=.01)
    glm.fit(X_train, y_train)
    gbdt.fit(X_train, y_train)
    print(glm.score(X_test, y_test))
    print(gbdt.score(X_test, y_test))





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    0.35776189065725783
    0.42425183539869415




.. GENERATED FROM PYTHON SOURCE LINES 56-64

Rich visual representation of estimators
-----------------------------------------
Estimators can now be visualized in notebooks by enabling the
`display='diagram'` option. This is particularly useful to summarise the
structure of pipelines and other composite estimators, with interactivity to
provide detail.  Click on the example image below to expand Pipeline
elements.  See :ref:`visualizing_composite_estimators` for how you can use
this feature.

.. GENERATED FROM PYTHON SOURCE LINES 64-85

.. code-block:: default


    from sklearn import set_config
    from sklearn.pipeline import make_pipeline
    from sklearn.preprocessing import OneHotEncoder, StandardScaler
    from sklearn.impute import SimpleImputer
    from sklearn.compose import make_column_transformer
    from sklearn.linear_model import LogisticRegression
    set_config(display='diagram')

    num_proc = make_pipeline(SimpleImputer(strategy='median'), StandardScaler())

    cat_proc = make_pipeline(
        SimpleImputer(strategy='constant', fill_value='missing'),
        OneHotEncoder(handle_unknown='ignore'))

    preprocessor = make_column_transformer((num_proc, ('feat1', 'feat3')),
                                           (cat_proc, ('feat0', 'feat2')))

    clf = make_pipeline(preprocessor, LogisticRegression())
    clf






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>div.sk-top-container {color: black;background-color: white;}div.sk-toggleable {background-color: white;}label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.2em 0.3em;box-sizing: border-box;text-align: center;}div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}div.sk-estimator {font-family: monospace;background-color: #f0f8ff;margin: 0.25em 0.25em;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;}div.sk-estimator:hover {background-color: #d4ebff;}div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;}div.sk-item {z-index: 1;}div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;}div.sk-parallel-item {display: flex;flex-direction: column;position: relative;background-color: white;}div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}div.sk-parallel-item:only-child::after {width: 0;}div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0.2em;box-sizing: border-box;padding-bottom: 0.1em;background-color: white;position: relative;}div.sk-label label {font-family: monospace;font-weight: bold;background-color: white;display: inline-block;line-height: 1.2em;}div.sk-label-container {position: relative;z-index: 2;text-align: center;}div.sk-container {display: inline-block;position: relative;}</style><div class="sk-top-container"><div class="sk-container"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="e7015934-ef97-4a6e-b40e-bd031d0e9320" type="checkbox" ><label class="sk-toggleable__label" for="e7015934-ef97-4a6e-b40e-bd031d0e9320">Pipeline</label><div class="sk-toggleable__content"><pre>Pipeline(steps=[('columntransformer',
                     ColumnTransformer(transformers=[('pipeline-1',
                                                      Pipeline(steps=[('simpleimputer',
                                                                       SimpleImputer(strategy='median')),
                                                                      ('standardscaler',
                                                                       StandardScaler())]),
                                                      ('feat1', 'feat3')),
                                                     ('pipeline-2',
                                                      Pipeline(steps=[('simpleimputer',
                                                                       SimpleImputer(fill_value='missing',
                                                                                     strategy='constant')),
                                                                      ('onehotencoder',
                                                                       OneHotEncoder(handle_unknown='ignore'))]),
                                                      ('feat0', 'feat2'))])),
                    ('logisticregression', LogisticRegression())])</pre></div></div></div><div class="sk-serial"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="bdcd29fb-75f1-4ba1-8b80-ec3563d9a7ba" type="checkbox" ><label class="sk-toggleable__label" for="bdcd29fb-75f1-4ba1-8b80-ec3563d9a7ba">columntransformer: ColumnTransformer</label><div class="sk-toggleable__content"><pre>ColumnTransformer(transformers=[('pipeline-1',
                                     Pipeline(steps=[('simpleimputer',
                                                      SimpleImputer(strategy='median')),
                                                     ('standardscaler',
                                                      StandardScaler())]),
                                     ('feat1', 'feat3')),
                                    ('pipeline-2',
                                     Pipeline(steps=[('simpleimputer',
                                                      SimpleImputer(fill_value='missing',
                                                                    strategy='constant')),
                                                     ('onehotencoder',
                                                      OneHotEncoder(handle_unknown='ignore'))]),
                                     ('feat0', 'feat2'))])</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="51b6e997-7422-4a46-a02e-780a0dd921e3" type="checkbox" ><label class="sk-toggleable__label" for="51b6e997-7422-4a46-a02e-780a0dd921e3">pipeline-1</label><div class="sk-toggleable__content"><pre>('feat1', 'feat3')</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="f67553ac-5401-4c20-a99c-950cf408d18b" type="checkbox" ><label class="sk-toggleable__label" for="f67553ac-5401-4c20-a99c-950cf408d18b">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='median')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="f1384049-7b08-4e39-83c9-ded116d9a38c" type="checkbox" ><label class="sk-toggleable__label" for="f1384049-7b08-4e39-83c9-ded116d9a38c">StandardScaler</label><div class="sk-toggleable__content"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="21dc2e3a-ab38-4660-a11c-4a70e3158a1d" type="checkbox" ><label class="sk-toggleable__label" for="21dc2e3a-ab38-4660-a11c-4a70e3158a1d">pipeline-2</label><div class="sk-toggleable__content"><pre>('feat0', 'feat2')</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="dfe77413-895e-4b67-910c-e73deb9e2114" type="checkbox" ><label class="sk-toggleable__label" for="dfe77413-895e-4b67-910c-e73deb9e2114">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(fill_value='missing', strategy='constant')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="37ba0700-e881-436c-9d33-ae82751213df" type="checkbox" ><label class="sk-toggleable__label" for="37ba0700-e881-436c-9d33-ae82751213df">OneHotEncoder</label><div class="sk-toggleable__content"><pre>OneHotEncoder(handle_unknown='ignore')</pre></div></div></div></div></div></div></div></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="7a3bbaea-433b-41b9-bcec-bce3396cc47f" type="checkbox" ><label class="sk-toggleable__label" for="7a3bbaea-433b-41b9-bcec-bce3396cc47f">LogisticRegression</label><div class="sk-toggleable__content"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 86-94

Scalability and stability improvements to KMeans
------------------------------------------------
The :class:`~sklearn.cluster.KMeans` estimator was entirely re-worked, and it
is now significantly faster and more stable. In addition, the Elkan algorithm
is now compatible with sparse matrices. The estimator uses OpenMP based
parallelism instead of relying on joblib, so the `n_jobs` parameter has no
effect anymore. For more details on how to control the number of threads,
please refer to our :ref:`parallelism` notes.

.. GENERATED FROM PYTHON SOURCE LINES 94-108

.. code-block:: default

    import scipy
    import numpy as np
    from sklearn.model_selection import train_test_split
    from sklearn.cluster import KMeans
    from sklearn.datasets import make_blobs
    from sklearn.metrics import completeness_score

    rng = np.random.RandomState(0)
    X, y = make_blobs(random_state=rng)
    X = scipy.sparse.csr_matrix(X)
    X_train, X_test, _, y_test = train_test_split(X, y, random_state=rng)
    kmeans = KMeans(algorithm='elkan').fit(X_train)
    print(completeness_score(kmeans.predict(X_test), y_test))





.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    0.7281859673451726




.. GENERATED FROM PYTHON SOURCE LINES 109-123

Improvements to the histogram-based Gradient Boosting estimators
----------------------------------------------------------------
Various improvements were made to
:class:`~sklearn.ensemble.HistGradientBoostingClassifier` and
:class:`~sklearn.ensemble.HistGradientBoostingRegressor`. On top of the
Poisson loss mentionned above, these estimators now support :ref:`sample
weights <sw_hgbdt>`. Also, an automatic early-stopping criterion was added:
early-stopping is enabled by default when the number of samples exceeds 10k.
Finally, users can now define :ref:`monotonic constraints
<monotonic_cst_gbdt>` to constrain the predictions based on the variations of
specific features. In the following example, we construct a target that is
generally positively correlated with the first feature, with some noise.
Applying monotoinc constraints allows the prediction to capture the global
effect of the first feature, instead of fitting the noise.

.. GENERATED FROM PYTHON SOURCE LINES 123-149

.. code-block:: default

    import numpy as np
    from matplotlib import pyplot as plt
    from sklearn.model_selection import train_test_split
    from sklearn.inspection import plot_partial_dependence
    from sklearn.experimental import enable_hist_gradient_boosting  # noqa
    from sklearn.ensemble import HistGradientBoostingRegressor

    n_samples = 500
    rng = np.random.RandomState(0)
    X = rng.randn(n_samples, 2)
    noise = rng.normal(loc=0.0, scale=0.01, size=n_samples)
    y = (5 * X[:, 0] + np.sin(10 * np.pi * X[:, 0]) - noise)

    gbdt_no_cst = HistGradientBoostingRegressor().fit(X, y)
    gbdt_cst = HistGradientBoostingRegressor(monotonic_cst=[1, 0]).fit(X, y)

    disp = plot_partial_dependence(
        gbdt_no_cst, X, features=[0], feature_names=['feature 0'],
        line_kw={'linewidth': 4, 'label': 'unconstrained'})
    plot_partial_dependence(gbdt_cst, X, features=[0],
        line_kw={'linewidth': 4, 'label': 'constrained'}, ax=disp.axes_)
    disp.axes_[0, 0].plot(X[:, 0], y, 'o', alpha=.5, zorder=-1, label='samples')
    disp.axes_[0, 0].set_ylim(-3, 3); disp.axes_[0, 0].set_xlim(-1, 1)
    plt.legend()
    plt.show()




.. image-sg:: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_0_23_0_001.png
   :alt: plot release highlights 0 23 0
   :srcset: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_0_23_0_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 150-154

Sample-weight support for Lasso and ElasticNet
----------------------------------------------
The two linear regressors :class:`~sklearn.linear_model.Lasso` and
:class:`~sklearn.linear_model.ElasticNet` now support sample weights.

.. GENERATED FROM PYTHON SOURCE LINES 154-169

.. code-block:: default


    from sklearn.model_selection import train_test_split
    from sklearn.datasets import make_regression
    from sklearn.linear_model import Lasso
    import numpy as np

    n_samples, n_features = 1000, 20
    rng = np.random.RandomState(0)
    X, y = make_regression(n_samples, n_features, random_state=rng)
    sample_weight = rng.rand(n_samples)
    X_train, X_test, y_train, y_test, sw_train, sw_test = train_test_split(
        X, y, sample_weight, random_state=rng)
    reg = Lasso()
    reg.fit(X_train, y_train, sample_weight=sw_train)
    print(reg.score(X_test, y_test, sw_test))




.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none

    0.999791942438998





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.828 seconds)


.. _sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_0_23_0.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_release_highlights_0_23_0.py <plot_release_highlights_0_23_0.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_release_highlights_0_23_0.ipynb <plot_release_highlights_0_23_0.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
