o \iA@s>dZddlZddlmZddlmZddZGdd d ZdS) zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) squared_normcCsV|jd}t|r|jtj|df||fd|S|dddf|}|j|S)z/Compute the sandwich product X.T @ diag(W) @ X.rshapeN)rrissparseT dia_matrixtoarray)XW n_samplesWXr/var/www/www-root/data/www/176.119.141.140/sports-predictor/venv/lib/python3.10/site-packages/sklearn/linear_model/_linear_loss.py sandwich_dots  rc@seZdZdZddZdddZddZd d Zd d Z   dddZ   dddZ   dddZ     dddZ dddZ dS)LinearModelLossa General class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the average of per sample losses and includes a term for L2 regularization:: loss = 1 / s_sum * sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None and s_sum=sum_i s_i. Gradient and hessian, for simplicity without intercept, are:: gradient = 1 / s_sum * X.T @ loss.gradient + l2_reg_strength * coef hessian = 1 / s_sum * X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_features::n_dof] = coef[(n_dof-1)::n_dof] intercept.shape = (n_classes,) else: intercept = coef[-1] Shape of gradient follows shape of coef. gradient.shape = coef.shape But hessian (to make our lives simpler) are always 2-d: if base_loss.is_multiclass: hessian.shape = (n_classes * n_dof, n_classes * n_dof) else: hessian.shape = (n_dof, n_dof) Note: If coef has shape (n_classes * n_dof,), the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool cCs||_||_dSN) base_loss fit_intercept)selfrrrrr__init__hs zLinearModelLoss.__init__NcCs\|jd}|jj}|jr|d}n|}|jjr$tj|||f|dd}|Stj|||d}|S)aAllocate coef of correct shape with zeros. Parameters: ----------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. dtype : data-type, default=None Overrides the data type of coef. With dtype=None, coef will have the same dtype as X. Returns ------- coef : ndarray of shape (n_dof,) or (n_classes, n_dof) Coefficients of a linear model. F)rdtypeorder)rr)rr n_classesr is_multiclassnp zeros_like)rr r n_featuresrn_dofcoefrrrinit_zero_coefls  zLinearModelLoss.init_zero_coefcCs|jjs|jr|d}|dd}||fSd}|}||fS|jdkr.|j|jjdfdd}n|}|jrI|dddf}|ddddf}||fSd}||fS)aHelper function to get coefficients and intercept. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. Nrrr)rrrndimreshaper)rr" interceptweightsrrrweight_intercepts"  z LinearModelLoss.weight_interceptcCs<||\}}|jjs|||}n||j|}|||fS)aiHelper function to get coefficients, intercept and raw_prediction. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. raw_prediction : ndarray of shape (n_samples,) or (n_samples, n_classes) )r+rrr)rr"r r*r)raw_predictionrrrweight_intercept_raws  z$LinearModelLoss.weight_intercept_rawcCs&|jdkr ||nt|}d||S)z5Compute L2 penalty term l2_reg_strength/2 *||w||_2^2.rg?)r'r)rr*l2_reg_strengthnorm2_wrrr l2_penaltys zLinearModelLoss.l2_penaltyr%rc Cs\|dur|||\}} }n||\}} |jj||d|d} tj| |d} | |||S)aCompute the loss as weighted average over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. Ny_truer, sample_weight n_threadsr*)r-r+rlossraverager0) rr"r yr3r.r4r,r*r)r6rrrr6s'zLinearModelLoss.losscCs^|j|jj\}} } | t|j} |dur|||\} } }n||\} } |jj||||d\}}|dur8|nt |}| |}|| | |7}||}|jj sutj || j d}|j||| |d| <|jrq| |d<||fStj| | f| j dd}|j||| |ddd| f<|jr|j dd|dddf<|jd kr|jdd }||fS) a\Computes the sum of loss and gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. gradient : ndarray of shape coef.shape The gradient of the loss. Nr1rr$rrrraxisrr&)rrrintrr-r+ loss_gradientrsumr0r empty_likerremptyr'ravel)rr"r r8r3r.r4r,r r rr!r*r)r6grad_pointwisesw_sumgradrrrr> s8*    "  zLinearModelLoss.loss_gradientcCs6|j|jj\}} } | t|j} |dur|||\} } }n||\} } |jj||||d}|dur6|nt |}||}|jj sctj || j d}|j ||| |d| <|jra| |d<|Stj| | f| j dd}|j ||| |ddd| f<|jr|j dd|dddf<|jd kr|jdd S|S) aComputes the gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. Nr1r9r$rr:rr;rr&)rrrr=rr-r+gradientrr?rr@rrrAr'rB)rr"r r8r3r.r4r,r r rr!r*r)rCrDrErrrrFXs4' "  zLinearModelLoss.gradientc Cs*|j|jj\} } } | t|j} | dur|||\}}} n||\}}|dur,| nt|}|dur?tj ||j dd}n!|j|jkrRt d|jd|jd|jj r^|j js^t d|}|j}|durrtj||f|j d}n'|j||fkrt d ||fd |jd|jj r|j js|j jst d |}|jj s.|jj|| ||d \}}||}||}tj|d k|ddk}t|}|j||||d| <|jr||d<|r|||fSt|||d| d| f<|d kr |j jrdnd}|jd|dd| | | d|7<|jr,|j|}||dddf<||dddf<||d<nb|jj|| ||d \}}||}|j| | fdd}|j||||ddd| f<|jrh|jd d|dddf<|jdkrt|jdd}|dur~||}nd|}t| D]}|dd|fd|dd|f|}t||||| | | || | | f<|jr|j|}|||| | | | | |f<||| | ||| | | f<||| | || | |f<t|d| D]v}|dd|f |dd|f|}t||||| | | || | | f<|jrL|j|}|||| | | | | |f<||| | ||| | | f<||| | || | |f<||d| |d| f||d| |d| f<qq|d kr|j jrrdnd}|jd|dd| d| | | | d|7<d}|||fS)a~Computes gradient and hessian w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. gradient_out : None or ndarray of shape coef.shape A location into which the gradient is stored. If None, a new array might be created. hessian_out : None or ndarray of shape (n_dof, n_dof) or (n_classes * n_dof, n_classes * n_dof) A location into which the hessian is stored. If None, a new array might be created. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessian : ndarray of shape (n_dof, n_dof) or (n_classes, n_dof, n_dof, n_classes) Hessian matrix. hessian_warning : bool True if pointwise hessian has more than 25% of its elements non-positive. Nrr:z4gradient_out is required to have shape coef.shape = z; got .z"gradient_out must be F-contiguous.r9z'hessian_out is required to have shape (z); got hessian_out.shape=zhessian_out must be contiguous.r1rr5g?r$Cr&r)r$r$r;g?rF)rrrr=rr-r+rr?r@r ValueErrorrflags f_contiguoussizerA c_contiguousgradient_hessianr7absrrr(gradient_probar'rBrange)rr"r r8r3r.r4 gradient_out hessian_outr,r r rr!r*r)rDrEnhessrChess_pointwisehessian_warningrXhprobaswkhlrrrrNs7             "   &  (       &       0   z LinearModelLoss.gradient_hessianc sj jj\}t j \ }} dur |nt  jjs jj || |d\} } | } | } tj  j d} j |  | d< jr[| | d<| t rrt j| df||fdn | ddtjf jrttjddt fdd } | | fS jj|| |d\} | } tjf j d d } | j  | dddf< jr| jdd| dddf< f d d } jd kr| jd d| fS| | fS)aComputes gradient and hessp (hessian product function) w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessp : callable Function that takes in a vector input of shape of gradient and and returns matrix-vector product with hessian. Nr1r9r$rrr;cst|}trj|d|d<ntjj|dg|d<|d|d7<jr]|d|d7<|d|d|d<|S)Nr$)rr@rrrlinalg multi_dotr)sret)r hXhX_sum hessian_sumr.r rrrhessps   $  z7LinearModelLoss.gradient_hessian_product..hessprr:cs|jdfdd}jr|dddf}|ddddf}nd}|j|}| |jddddtjf7}|9}durM|ddtjf9}tjf jdd}|j ||dddf<jr||jdd |dddf<jdkr|j ddS|S)Nr$rr&rrr;r:) r(rrr?rnewaxisrArr'rB)r` s_intercepttmp hess_prod) r r"r.rr!r rYr3rrDr*rrres"$&  rr&)rrrr=rr-rr?rrNr@rrrrr rfsqueezeasarray atleast_1drPrAr'rB)rr"r r8r3r.r4r r)r,rCrVrErer)r r"rbrcrdr.rr!r rYr3rrDr*rgradient_hessian_productsX     N "  z(LinearModelLoss.gradient_hessian_productr)Nr%rN)Nr%rNNN)Nr%r)__name__ __module__ __qualname____doc__rr#r+r-r0r6r>rFrNrmrrrrr%sBB '   ; S N r) rqnumpyrscipyr utils.extmathrrrrrrrs