Blog

Scikit-Learn : LASSO

Scikit-Learn : LASSO

LASSO (Least Absolute Shrinkage and Selection Operator)

LASSO is the regularisation technique that performs L1 regularisation. It modifies the loss function by adding the penalty (shrinkage quantity) equivalent to the summation of the absolute value of coefficients.βˆ‘j=1m(Yiβˆ’W0βˆ’βˆ‘i=1nWiXji)2+Ξ±βˆ‘i=1n|Wi|=lossβˆ’function+Ξ±βˆ‘i=1n|Wi|

sklearn.linear_model. Lasso is a linear model, with an added regularisation term, used to estimate sparse coefficients.

Parameters

Followings table consist the parameters used by Lasso module βˆ’

Sr.NoParameter & Description
1alpha βˆ’ float, optional, default = 1.0Alpha, the constant that multiplies the L1 term, is the tuning parameter that decides how much we want to penalize the model. The default value is 1.0.
2fit_intercept βˆ’ Boolean, optional. Default=TrueThis parameter specifies that a constant (bias or intercept) should be added to the decision function. No intercept will be used in calculation, if it will set to false.
3tol βˆ’ float, optionalThis parameter represents the tolerance for the optimization. The tol value and updates would be compared and if found updates smaller than tol, the optimization checks the dual gap for optimality and continues until it is smaller than tol.
4normalize βˆ’ Boolean, optional, default = FalseIf this parameter is set to True, the regressor X will be normalized before regression. The normalization will be done by subtracting the mean and dividing it by L2 norm. If fit_intercept = False, this parameter will be ignored.
5copy_X βˆ’ Boolean, optional, default = TrueBy default, it is true which means X will be copied. But if it is set to false, X may be overwritten.
6max_iter βˆ’ int, optionalAs name suggest, it represents the maximum number of iterations taken for conjugate gradient solvers.
7precompute βˆ’ True|False|array-like, default=FalseWith this parameter we can decide whether to use a precomputed Gram matrix to speed up the calculation or not.
8warm_start βˆ’ bool, optional, default = falseWith this parameter set to True, we can reuse the solution of the previous call to fit as initialization. If we choose default i.e. false, it will erase the previous solution.
9random_state βˆ’ int, RandomState instance or None, optional, default = noneThis parameter represents the seed of the pseudo random number generated which is used while shuffling the data. Followings are the options βˆ’int βˆ’ In this case, random_state is the seed used by random number generator.RandomState instance βˆ’ In this case, random_state is the random number generator.None βˆ’ In this case, the random number generator is the RandonState instance used by np.random.
10selection βˆ’ str, default=β€˜cyclic’Cyclic βˆ’ The default value is cyclic which means the features will be looping over sequentially by default.Random βˆ’ If we set the selection to random, a random coefficient will be updated every iteration.

Attributes

Followings table consist the attributes used by Lasso module βˆ’

Sr.NoAttributes & Description
1coef_ βˆ’ array, shape(n_features,) or (n_target, n_features)This attribute provides the weight vectors.
2Intercept_ βˆ’ float | array, shape = (n_targets)It represents the independent term in decision function.
3n_iter_ βˆ’ int or array-like, shape (n_targets)It gives the number of iterations run by the coordinate descent solver to reach the specified tolerance.

Implementation Example

Following Python script uses Lasso model which further uses coordinate descent as the algorithm to fit the coefficients βˆ’

from sklearn import linear_model
Lreg = linear_model.Lasso(alpha = 0.5)
Lreg.fit([[0,0], [1, 1], [2, 2]], [0, 1, 2])

Output

Lasso(alpha = 0.5, copy_X = True, fit_intercept = True, max_iter = 1000,
   normalize = False, positive = False, precompute = False, random_state = None,
   selection = 'cyclic', tol = 0.0001, warm_start = False)

Example

Now, once fitted, the model can predict new values as follows βˆ’

Lreg.predict([[0,1]])

Output

array([0.75])

Example

For the above example, we can get the weight vector with the help of following python script βˆ’

Lreg.coef_

Output

array([0.25, 0. ])

Example

Similarly, we can get the value of intercept with the help of following python script βˆ’

Lreg.intercept_

Output

0.75

Example

We can get the total number of iterations to get the specified tolerance with the help of following python script βˆ’

Lreg.n_iter_

Output

2

We can change the values of parameters to get the desired output from the model.

Next Topic : Click Here

This Post Has 2 Comments

Leave a Reply