o \ix:@sdZddlZddlmZmZddlZddlmZ ddl m Z ddl m Z mZmZmZddlmZmZddlmZdd lmZdd lmZmZGd d d eee ZdS) zRestricted Boltzmann MachineN)IntegralReal)expit) BaseEstimatorClassNamePrefixFeaturesOutMixinTransformerMixin _fit_context)check_random_stategen_even_slices)Interval)safe_sparse_dot)check_is_fitted validate_datac seZdZUdZeeddddgeeddddgeeddddgeeddddgdgd gd Zee d < d*d dddddddZ ddZ ddZ ddZ ddZddZddZeddd+d d!Zd"d#Zd$d%Zeddd+d&d'Zfd(d)ZZS), BernoulliRBMaBernoulli Restricted Boltzmann Machine (RBM). A Restricted Boltzmann Machine with binary visible units and binary hidden units. Parameters are estimated using Stochastic Maximum Likelihood (SML), also known as Persistent Contrastive Divergence (PCD) [2]. The time complexity of this implementation is ``O(d ** 2)`` assuming d ~ n_features ~ n_components. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=256 Number of binary hidden units. learning_rate : float, default=0.1 The learning rate for weight updates. It is *highly* recommended to tune this hyper-parameter. Reasonable values are in the 10**[0., -3.] range. batch_size : int, default=10 Number of examples per minibatch. n_iter : int, default=10 Number of iterations/sweeps over the training dataset to perform during training. verbose : int, default=0 The verbosity level. The default, zero, means silent mode. Range of values is [0, inf]. random_state : int, RandomState instance or None, default=None Determines random number generation for: - Gibbs sampling from visible and hidden layers. - Initializing components, sampling from layers during fit. - Corrupting the data when scoring samples. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. Attributes ---------- intercept_hidden_ : array-like of shape (n_components,) Biases of the hidden units. intercept_visible_ : array-like of shape (n_features,) Biases of the visible units. components_ : array-like of shape (n_components, n_features) Weight matrix, where `n_features` is the number of visible units and `n_components` is the number of hidden units. h_samples_ : array-like of shape (batch_size, n_components) Hidden Activation sampled from the model distribution, where `batch_size` is the number of examples per minibatch and `n_components` is the number of hidden units. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- sklearn.neural_network.MLPRegressor : Multi-layer Perceptron regressor. sklearn.neural_network.MLPClassifier : Multi-layer Perceptron classifier. sklearn.decomposition.PCA : An unsupervised linear dimensionality reduction model. References ---------- [1] Hinton, G. E., Osindero, S. and Teh, Y. A fast learning algorithm for deep belief nets. Neural Computation 18, pp 1527-1554. https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf [2] Tieleman, T. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. International Conference on Machine Learning (ICML) 2008 Examples -------- >>> import numpy as np >>> from sklearn.neural_network import BernoulliRBM >>> X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]]) >>> model = BernoulliRBM(n_components=2) >>> model.fit(X) BernoulliRBM(n_components=2) For a more detailed example usage, see :ref:`sphx_glr_auto_examples_neural_networks_plot_rbm_logistic_classification.py`. Nleft)closedrneitherverbose random_state n_components learning_rate batch_sizen_iterrr_parameter_constraintsg? )rrrrrcCs(||_||_||_||_||_||_dSNr)selfrrrrrrr!|/var/www/www-root/data/www/176.119.141.140/sports-predictor/venv/lib/python3.10/site-packages/sklearn/neural_network/_rbm.py__init__s   zBernoulliRBM.__init__cCs,t|t||ddtjtjfd}||S)agCompute the hidden layer activation probabilities, P(h=1|v=X). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The data to be transformed. Returns ------- h : ndarray of shape (n_samples, n_components) Latent representations of the data. csrF) accept_sparseresetdtype)rrnpfloat64float32 _mean_hiddens)r Xr!r!r" transforms   zBernoulliRBM.transformcCs$t||jj}||j7}t||dS)aLComputes the probabilities P(h=1|v). Parameters ---------- v : ndarray of shape (n_samples, n_features) Values of the visible layer. Returns ------- h : ndarray of shape (n_samples, n_components) Corresponding mean field values for the hidden layer. out)r components_Tintercept_hidden_r)r vpr!r!r"r+s  zBernoulliRBM._mean_hiddenscCs||}|j|jd|kS)aSample from the distribution P(h|v). Parameters ---------- v : ndarray of shape (n_samples, n_features) Values of the visible layer to sample from. rng : RandomState instance Random number generator to use. Returns ------- h : ndarray of shape (n_samples, n_components) Values of the hidden layer. size)r+uniformshape)r r3rngr4r!r!r"_sample_hiddenss zBernoulliRBM._sample_hiddenscCs6t||j}||j7}t||d|j|jd|kS)aSample from the distribution P(v|h). Parameters ---------- h : ndarray of shape (n_samples, n_components) Values of the hidden layer to sample from. rng : RandomState instance Random number generator to use. Returns ------- v : ndarray of shape (n_samples, n_features) Values of the visible layer. r.r5)r(dotr0intercept_visible_rr7r8)r hr9r4r!r!r"_sample_visibless  zBernoulliRBM._sample_visiblescCs2t||j tdt||jj|jjddS)aFComputes the free energy F(v) = - log sum_h exp(-E(v,h)). Parameters ---------- v : ndarray of shape (n_samples, n_features) Values of the visible layer. Returns ------- free_energy : ndarray of shape (n_samples,) The value of the free energy. rraxis)r r<r( logaddexpr0r1r2sum)r r3r!r!r" _free_energys  zBernoulliRBM._free_energycCs>t|t|dst|j|_|||j}|||j}|S)aTPerform one Gibbs sampling step. Parameters ---------- v : ndarray of shape (n_samples, n_features) Values of the visible layer to start from. Returns ------- v_new : ndarray of shape (n_samples, n_features) Values of the visible layer after one Gibbs step. random_state_)rhasattrr rrDr:r>)r r3h_v_r!r!r"gibbss   zBernoulliRBM.gibbsT)prefer_skip_nested_validationcCst|d }t||dtj|d}t|dst|j|_t|ds;tj|jdd|j |j dfdd |_ |j j d|_ t|d sGt |j |_t|d sUt |j d|_t|d sdt |j|j f|_|||jd S)aFit the model to the partial segment of the data X. Parameters ---------- X : ndarray of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) or (n_samples, n_outputs), default=None Target values (None for unsupervised transformations). Returns ------- self : BernoulliRBM The fitted model. r0r$)r%r'r&rDr{Gz?rF)orderr2r< h_samples_N)rErr(r)r rrDasarraynormalrr8r0_n_features_outzerosr2r<rrM_fit)r r,y first_passr!r!r" partial_fits.        zBernoulliRBM.partial_fitcCs||}||j|}||}t|j|jd}t|j|ddj}|t |j|8}|j ||7_ |j ||j dd|j dd7_ |j |t|j dd|j dd7_ d||j|jd|k<t|||_dS)aInner fit for one mini-batch. Adjust the parameters to maximize the likelihood of v using Stochastic Maximum Likelihood (SML). Parameters ---------- v_pos : ndarray of shape (n_samples, n_features) The data to use for training. rng : RandomState instance Random number generator to use for sampling. rT) dense_outputr?g?r5N)r+r>rMfloatrr8r r1r(r;r0r2rBr<rNsqueezer7floor)r v_posr9h_posv_negh_neglrupdater!r!r"rR:s  & zBernoulliRBM._fitc Cst|t||ddd}t|j}t|jd|d|jd|jdf}t |rXd||d}t |tj rI|tj |j |f|jd}n|tj||f|jd}n |}d||||<||}||}|jd td|| S)a|Compute the pseudo-likelihood of X. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Values of the visible layer. Must be all-boolean (not checked). Returns ------- pseudo_likelihood : ndarray of shape (n_samples,) Value of the pseudo-likelihood (proxy for likelihood). Notes ----- This method is not deterministic: it computes a quantity called the free energy on X, then on a randomly corrupted version of X, and returns the log of the logistic function of the difference. r$F)r%r&rr)r8)rrr rr(aranger8randintspissparse isinstancematrix csr_matrixAravel csr_arraycopyrCrA) r r,r3r9inddatarGfefe_r!r!r" score_samplesXs *     zBernoulliRBM.score_samplesc CsTt||dtjtjfd}|jd}t|j}tj|dd|j |jdfd|j d|_ |j jd|_ tj |j |j d|_tj |jd|j d|_tj |j|j f|j d|_ttt||j}tt||j||d }|j}t}td|jdD]+} |D] } ||| |q|rt} td t|j| || | |f| }q||S) aFit the model to the data X. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) or (n_samples, n_outputs), default=None Target values (None for unsupervised transformations). Returns ------- self : BernoulliRBM The fitted model. r$)r%r'rrJrrK)rLr')r') n_samplesz9[%s] Iteration %d, pseudo-likelihood = %.2f, time = %.2fs)!rr(r)r*r8r rrNrOrr'r0rPrQr2r<rrMintceilrWlistr rtimerangerrRprinttype__name__rpmean) r r,rSrqr9 n_batches batch_slicesrbegin iteration batch_sliceendr!r!r"fitsF    zBernoulliRBM.fitcs"t}d|j_ddg|j_|S)NTr)r*)super__sklearn_tags__ input_tagssparsetransformer_tagspreserves_dtype)r tags __class__r!r"rs  zBernoulliRBM.__sklearn_tags__)rr)ry __module__ __qualname____doc__r rrrdict__annotations__r#r-r+r:r>rCrHr rUrRrprr __classcell__r!r!rr"rs< j   )) 7r)rrunumbersrrnumpyr( scipy.sparserrc scipy.specialrbaserrrr utilsr r utils._param_validationr utils.extmathr utils.validationrrrr!r!r!r"s