
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/manifold/plot_lle_digits.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_examples_manifold_plot_lle_digits.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_manifold_plot_lle_digits.py:


=============================================================================
Manifold learning on handwritten digits: Locally Linear Embedding, Isomap...
=============================================================================

An illustration of various embeddings on the digits dataset.

The RandomTreesEmbedding, from the :mod:`sklearn.ensemble` module, is not
technically a manifold embedding method, as it learn a high-dimensional
representation on which we apply a dimensionality reduction method.
However, it is often useful to cast a dataset into a representation in
which the classes are linearly-separable.

t-SNE will be initialized with the embedding that is generated by PCA in
this example, which is not the default setting. It ensures global stability
of the embedding, i.e., the embedding does not depend on random
initialization.

Linear Discriminant Analysis, from the :mod:`sklearn.discriminant_analysis`
module, and Neighborhood Components Analysis, from the :mod:`sklearn.neighbors`
module, are supervised dimensionality reduction method, i.e. they make use of
the provided labels, contrary to other methods.

.. GENERATED FROM PYTHON SOURCE LINES 24-249



.. rst-class:: sphx-glr-horizontal


    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_001.png
         :alt: A selection from the 64-dimensional digits dataset
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_001.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_002.png
         :alt: Random Projection of the digits
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_002.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_003.png
         :alt: Principal Components projection of the digits (time 0.00s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_003.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_004.png
         :alt: Linear Discriminant projection of the digits (time 0.01s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_004.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_005.png
         :alt: Isomap projection of the digits (time 1.00s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_005.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_006.png
         :alt: Locally Linear Embedding of the digits (time 0.26s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_006.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_007.png
         :alt: Modified Locally Linear Embedding of the digits (time 0.55s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_007.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_008.png
         :alt: Hessian Locally Linear Embedding of the digits (time 0.60s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_008.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_009.png
         :alt: Local Tangent Space Alignment of the digits (time 0.51s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_009.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_010.png
         :alt: MDS embedding of the digits (time 1.17s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_010.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_011.png
         :alt: Random forest embedding of the digits (time 0.18s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_011.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_012.png
         :alt: Spectral embedding of the digits (time 0.27s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_012.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_013.png
         :alt: t-SNE embedding of the digits (time 2.52s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_013.png
         :class: sphx-glr-multi-img

    *

      .. image-sg:: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_014.png
         :alt: NCA embedding of the digits (time 2.12s)
         :srcset: /auto_examples/manifold/images/sphx_glr_plot_lle_digits_014.png
         :class: sphx-glr-multi-img


.. rst-class:: sphx-glr-script-out

 Out:

 .. code-block:: none


    Computing random projection
    Computing PCA projection
    Computing Linear Discriminant Analysis projection
    Computing Isomap projection
    Done.
    Computing LLE embedding
    Done. Reconstruction error: 1.63545e-06
    Computing modified LLE embedding
    Done. Reconstruction error: 0.360357
    Computing Hessian LLE embedding
    Done. Reconstruction error: 0.212804
    Computing LTSA embedding
    Done. Reconstruction error: 0.212807
    Computing MDS embedding
    Done. Stress: 140428674.316512
    Computing Totally Random Trees embedding
    Computing Spectral embedding
    Computing t-SNE embedding
    Computing NCA projection






|

.. code-block:: default


    # Authors: Fabian Pedregosa <fabian.pedregosa@inria.fr>
    #          Olivier Grisel <olivier.grisel@ensta.org>
    #          Mathieu Blondel <mathieu@mblondel.org>
    #          Gael Varoquaux
    # License: BSD 3 clause (C) INRIA 2011

    from time import time
    import numpy as np
    import matplotlib.pyplot as plt
    from matplotlib import offsetbox
    from sklearn import (manifold, datasets, decomposition, ensemble,
                         discriminant_analysis, random_projection, neighbors)
    print(__doc__)

    digits = datasets.load_digits(n_class=6)
    X = digits.data
    y = digits.target
    n_samples, n_features = X.shape
    n_neighbors = 30


    # ----------------------------------------------------------------------
    # Scale and visualize the embedding vectors
    def plot_embedding(X, title=None):
        x_min, x_max = np.min(X, 0), np.max(X, 0)
        X = (X - x_min) / (x_max - x_min)

        plt.figure()
        ax = plt.subplot(111)
        for i in range(X.shape[0]):
            plt.text(X[i, 0], X[i, 1], str(y[i]),
                     color=plt.cm.Set1(y[i] / 10.),
                     fontdict={'weight': 'bold', 'size': 9})

        if hasattr(offsetbox, 'AnnotationBbox'):
            # only print thumbnails with matplotlib > 1.0
            shown_images = np.array([[1., 1.]])  # just something big
            for i in range(X.shape[0]):
                dist = np.sum((X[i] - shown_images) ** 2, 1)
                if np.min(dist) < 4e-3:
                    # don't show points that are too close
                    continue
                shown_images = np.r_[shown_images, [X[i]]]
                imagebox = offsetbox.AnnotationBbox(
                    offsetbox.OffsetImage(digits.images[i], cmap=plt.cm.gray_r),
                    X[i])
                ax.add_artist(imagebox)
        plt.xticks([]), plt.yticks([])
        if title is not None:
            plt.title(title)


    # ----------------------------------------------------------------------
    # Plot images of the digits
    n_img_per_row = 20
    img = np.zeros((10 * n_img_per_row, 10 * n_img_per_row))
    for i in range(n_img_per_row):
        ix = 10 * i + 1
        for j in range(n_img_per_row):
            iy = 10 * j + 1
            img[ix:ix + 8, iy:iy + 8] = X[i * n_img_per_row + j].reshape((8, 8))

    plt.imshow(img, cmap=plt.cm.binary)
    plt.xticks([])
    plt.yticks([])
    plt.title('A selection from the 64-dimensional digits dataset')


    # ----------------------------------------------------------------------
    # Random 2D projection using a random unitary matrix
    print("Computing random projection")
    rp = random_projection.SparseRandomProjection(n_components=2, random_state=42)
    X_projected = rp.fit_transform(X)
    plot_embedding(X_projected, "Random Projection of the digits")


    # ----------------------------------------------------------------------
    # Projection on to the first 2 principal components

    print("Computing PCA projection")
    t0 = time()
    X_pca = decomposition.TruncatedSVD(n_components=2).fit_transform(X)
    plot_embedding(X_pca,
                   "Principal Components projection of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # Projection on to the first 2 linear discriminant components

    print("Computing Linear Discriminant Analysis projection")
    X2 = X.copy()
    X2.flat[::X.shape[1] + 1] += 0.01  # Make X invertible
    t0 = time()
    X_lda = discriminant_analysis.LinearDiscriminantAnalysis(n_components=2
                                                             ).fit_transform(X2, y)
    plot_embedding(X_lda,
                   "Linear Discriminant projection of the digits (time %.2fs)" %
                   (time() - t0))


    # ----------------------------------------------------------------------
    # Isomap projection of the digits dataset
    print("Computing Isomap projection")
    t0 = time()
    X_iso = manifold.Isomap(n_neighbors=n_neighbors, n_components=2
                            ).fit_transform(X)
    print("Done.")
    plot_embedding(X_iso,
                   "Isomap projection of the digits (time %.2fs)" %
                   (time() - t0))


    # ----------------------------------------------------------------------
    # Locally linear embedding of the digits dataset
    print("Computing LLE embedding")
    clf = manifold.LocallyLinearEmbedding(n_neighbors=n_neighbors, n_components=2,
                                          method='standard')
    t0 = time()
    X_lle = clf.fit_transform(X)
    print("Done. Reconstruction error: %g" % clf.reconstruction_error_)
    plot_embedding(X_lle,
                   "Locally Linear Embedding of the digits (time %.2fs)" %
                   (time() - t0))


    # ----------------------------------------------------------------------
    # Modified Locally linear embedding of the digits dataset
    print("Computing modified LLE embedding")
    clf = manifold.LocallyLinearEmbedding(n_neighbors=n_neighbors, n_components=2,
                                          method='modified')
    t0 = time()
    X_mlle = clf.fit_transform(X)
    print("Done. Reconstruction error: %g" % clf.reconstruction_error_)
    plot_embedding(X_mlle,
                   "Modified Locally Linear Embedding of the digits (time %.2fs)" %
                   (time() - t0))


    # ----------------------------------------------------------------------
    # HLLE embedding of the digits dataset
    print("Computing Hessian LLE embedding")
    clf = manifold.LocallyLinearEmbedding(n_neighbors=n_neighbors, n_components=2,
                                          method='hessian')
    t0 = time()
    X_hlle = clf.fit_transform(X)
    print("Done. Reconstruction error: %g" % clf.reconstruction_error_)
    plot_embedding(X_hlle,
                   "Hessian Locally Linear Embedding of the digits (time %.2fs)" %
                   (time() - t0))


    # ----------------------------------------------------------------------
    # LTSA embedding of the digits dataset
    print("Computing LTSA embedding")
    clf = manifold.LocallyLinearEmbedding(n_neighbors=n_neighbors, n_components=2,
                                          method='ltsa')
    t0 = time()
    X_ltsa = clf.fit_transform(X)
    print("Done. Reconstruction error: %g" % clf.reconstruction_error_)
    plot_embedding(X_ltsa,
                   "Local Tangent Space Alignment of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # MDS  embedding of the digits dataset
    print("Computing MDS embedding")
    clf = manifold.MDS(n_components=2, n_init=1, max_iter=100)
    t0 = time()
    X_mds = clf.fit_transform(X)
    print("Done. Stress: %f" % clf.stress_)
    plot_embedding(X_mds,
                   "MDS embedding of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # Random Trees embedding of the digits dataset
    print("Computing Totally Random Trees embedding")
    hasher = ensemble.RandomTreesEmbedding(n_estimators=200, random_state=0,
                                           max_depth=5)
    t0 = time()
    X_transformed = hasher.fit_transform(X)
    pca = decomposition.TruncatedSVD(n_components=2)
    X_reduced = pca.fit_transform(X_transformed)

    plot_embedding(X_reduced,
                   "Random forest embedding of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # Spectral embedding of the digits dataset
    print("Computing Spectral embedding")
    embedder = manifold.SpectralEmbedding(n_components=2, random_state=0,
                                          eigen_solver="arpack")
    t0 = time()
    X_se = embedder.fit_transform(X)

    plot_embedding(X_se,
                   "Spectral embedding of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # t-SNE embedding of the digits dataset
    print("Computing t-SNE embedding")
    tsne = manifold.TSNE(n_components=2, init='pca', random_state=0)
    t0 = time()
    X_tsne = tsne.fit_transform(X)

    plot_embedding(X_tsne,
                   "t-SNE embedding of the digits (time %.2fs)" %
                   (time() - t0))

    # ----------------------------------------------------------------------
    # NCA projection of the digits dataset
    print("Computing NCA projection")
    nca = neighbors.NeighborhoodComponentsAnalysis(init='random',
                                                   n_components=2, random_state=0)
    t0 = time()
    X_nca = nca.fit_transform(X, y)

    plot_embedding(X_nca,
                   "NCA embedding of the digits (time %.2fs)" %
                   (time() - t0))

    plt.show()


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  19.745 seconds)


.. _sphx_glr_download_auto_examples_manifold_plot_lle_digits.py:


.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-example



  .. container:: sphx-glr-download sphx-glr-download-python

     :download:`Download Python source code: plot_lle_digits.py <plot_lle_digits.py>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

     :download:`Download Jupyter notebook: plot_lle_digits.ipynb <plot_lle_digits.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
