

.. _sphx_glr_auto_examples_linear_model_plot_sgd_penalties.py:


==============
SGD: Penalties
==============

Plot the contours of the three penalties.

All of the above are supported by
:class:`sklearn.linear_model.stochastic_gradient`.




.. code-block:: python

    from __future__ import division
    print(__doc__)

    import numpy as np
    import matplotlib.pyplot as plt


    def l1(xs):
        return np.array([np.sqrt((1 - np.sqrt(x ** 2.0)) ** 2.0) for x in xs])


    def l2(xs):
        return np.array([np.sqrt(1.0 - x ** 2.0) for x in xs])


    def el(xs, z):
        return np.array([(2 - 2 * x - 2 * z + 4 * x * z -
                          (4 * z ** 2
                           - 8 * x * z ** 2
                           + 8 * x ** 2 * z ** 2
                           - 16 * x ** 2 * z ** 3
                           + 8 * x * z ** 3 + 4 * x ** 2 * z ** 4) ** (1. / 2)
                          - 2 * x * z ** 2) / (2 - 4 * z) for x in xs])


    def cross(ext):
        plt.plot([-ext, ext], [0, 0], "k-")
        plt.plot([0, 0], [-ext, ext], "k-")

    xs = np.linspace(0, 1, 100)

    alpha = 0.501  # 0.5 division throuh zero

    cross(1.2)

    l1_color = "navy"
    l2_color = "c"
    elastic_net_color = "darkorange"
    lw = 2

    plt.plot(xs, l1(xs), color=l1_color, label="L1", lw=lw)
    plt.plot(xs, -1.0 * l1(xs), color=l1_color, lw=lw)
    plt.plot(-1 * xs, l1(xs), color=l1_color, lw=lw)
    plt.plot(-1 * xs, -1.0 * l1(xs), color=l1_color, lw=lw)

    plt.plot(xs, l2(xs), color=l2_color, label="L2", lw=lw)
    plt.plot(xs, -1.0 * l2(xs), color=l2_color, lw=lw)
    plt.plot(-1 * xs, l2(xs), color=l2_color, lw=lw)
    plt.plot(-1 * xs, -1.0 * l2(xs), color=l2_color, lw=lw)

    plt.plot(xs, el(xs, alpha), color=elastic_net_color, label="Elastic Net", lw=lw)
    plt.plot(xs, -1.0 * el(xs, alpha), color=elastic_net_color, lw=lw)
    plt.plot(-1 * xs, el(xs, alpha), color=elastic_net_color, lw=lw)
    plt.plot(-1 * xs, -1.0 * el(xs, alpha), color=elastic_net_color, lw=lw)

    plt.xlabel(r"$w_0$")
    plt.ylabel(r"$w_1$")
    plt.legend()

    plt.axis("equal")
    plt.show()

**Total running time of the script:** ( 0 minutes  0.000 seconds)



.. container:: sphx-glr-footer


  .. container:: sphx-glr-download

     :download:`Download Python source code: plot_sgd_penalties.py <plot_sgd_penalties.py>`



  .. container:: sphx-glr-download

     :download:`Download Jupyter notebook: plot_sgd_penalties.ipynb <plot_sgd_penalties.ipynb>`

.. rst-class:: sphx-glr-signature

    `Generated by Sphinx-Gallery <https://sphinx-gallery.readthedocs.io>`_
