The Elastic-Net is a regularised regression method that linearly combines both penalties i.e. L1 and L2 of the Lasso and Ridge regression methods. It is useful when there are multiple correlated features. The difference between Lass and Elastic-Net lies in the fact that Lasso is likely to pick one of these features at random while elastic-net is likely to pick both at once.
Sklearn provides a linear model named ElasticNet which is trained with both L1, L2-norm for regularisation of the coefficients. The advantage of such combination is that it allows for learning a sparse model where few of the weights are non-zero like Lasso regularisation method, while still maintaining the regularization properties of Ridge regularisation method.
Following is the objective function to minimise −minw12nsamples∥Xw−Y∥22+αρ∥W∥1+α⟮1−ρ⟯2 ∥W∥22
Parameters
Following table consist the parameters used by ElasticNet module −
| Sr.No | Parameter & Description |
|---|---|
| 1 | alpha − float, optional, default = 1.0Alpha, the constant that multiplies the L1/L2 term, is the tuning parameter that decides how much we want to penalize the model. The default value is 1.0. |
| 2 | l1_ratio − floatThis is called the ElasticNet mixing parameter. Its range is 0 < = l1_ratio < = 1. If l1_ratio = 1, the penalty would be L1 penalty. If l1_ratio = 0, the penalty would be an L2 penalty. If the value of l1 ratio is between 0 and 1, the penalty would be the combination of L1 and L2. |
| 3 | fit_intercept − Boolean, optional. Default=TrueThis parameter specifies that a constant (bias or intercept) should be added to the decision function. No intercept will be used in calculation, if it will set to false. |
| 4 | tol − float, optionalThis parameter represents the tolerance for the optimization. The tol value and updates would be compared and if found updates smaller than tol, the optimization checks the dual gap for optimality and continues until it is smaller than tol. |
| 5 | normalise − Boolean, optional, default = FalseIf this parameter is set to True, the regressor X will be normalised before regression. The normalisation will be done by subtracting the mean and dividing it by L2 norm. If fit_intercept = False, this parameter will be ignored. |
| 6 | precompute − True|False|array-like, default=FalseWith this parameter we can decide whether to use a precomputed Gram matrix to speed up the calculation or not. To preserve sparsity, it would always be true for sparse input. |
| 7 | copy_X − Boolean, optional, default = TrueBy default, it is true which means X will be copied. But if it is set to false, X may be overwritten. |
| 8 | max_iter − int, optionalAs name suggest, it represents the maximum number of iterations taken for conjugate gradient solvers. |
| 9 | warm_start − bool, optional, default = falseWith this parameter set to True, we can reuse the solution of the previous call to fit as initialisation. If we choose default i.e. false, it will erase the previous solution. |
| 10 | random_state − int, RandomState instance or None, optional, default = noneThis parameter represents the seed of the pseudo random number generated which is used while shuffling the data. Following are the options −int − In this case, random_state is the seed used by random number generator.RandomState instance − In this case, random_state is the random number generator.None − In this case, the random number generator is the RandonState instance used by np.random. |
| 11 | selection − str, default=‘cyclic’Cyclic − The default value is cyclic which means the features will be looping over sequentially by default.Random − If we set the selection to random, a random coefficient will be updated every iteration. |
Attributes
Followings table consist the attributes used by ElasticNet module −
| Sr.No | Attributes & Description |
|---|---|
| 1 | coef_ − array, shape (n_tasks, n_features)This attribute provides the weight vectors. |
| 2 | Intercept_ − array, shape (n_tasks)It represents the independent term in decision function. |
| 3 | n_iter_ − intIt gives the number of iterations run by the coordinate descent solver to reach the specified tolerance. |
Implementation Example
Following Python script uses ElasticNet linear model which further uses coordinate descent as the algorithm to fit the coefficients −
from sklearn import linear_model ENreg = linear_model.ElasticNet(alpha = 0.5,random_state = 0) ENreg.fit([[0,0], [1, 1], [2, 2]], [0, 1, 2])
Output
ElasticNet(alpha = 0.5, copy_X = True, fit_intercept = True, l1_ratio = 0.5, max_iter = 1000, normalize = False, positive = False, precompute=False, random_state = 0, selection = 'cyclic', tol = 0.0001, warm_start = False)
Example
Now, once fitted, the model can predict new values as follows −
ENregReg.predict([[0,1]])
Output
array([0.73686077])
Example
For the above example, we can get the weight vector with the help of following python script −
ENreg.coef_
Output
array([0.26318357, 0.26313923])
Example
Similarly, we can get the value of intercept with the help of following python script −
ENreg.intercept_
Output
0.47367720941913904
Example
We can get the total number of iterations to get the specified tolerance with the help of following python script −
ENreg.n_iter_
Output
15
We can change the values of alpha (towards 1) to get better results from the model.
Example
Let us see same example with alpha = 1.
from sklearn import linear_model ENreg = linear_model.ElasticNet(alpha = 1,random_state = 0) ENreg.fit([[0,0], [1, 1], [2, 2]], [0, 1, 2]) Output ElasticNet(alpha = 1, copy_X = True, fit_intercept = True, l1_ratio = 0.5, max_iter = 1000, normalize = False, positive = False, precompute = False, random_state = 0, selection = 'cyclic', tol = 0.0001, warm_start = False) #Predicting new values ENreg.predict([[1,0]]) Output array([0.90909216]) #weight vectors ENreg.coef_ Output array([0.09091128, 0.09090784]) #Calculating intercept ENreg.intercept_ Output 0.818180878658411 #Calculating number of iterations ENreg.n_iter_ Output 10
From the above examples, we can see the difference in the outputs.
Next Topic : Click Here
Pingback: Scikit-Learn : Linear Modeling | Adglob Infosystem Pvt Ltd
Pingback: Scikit-Learn : Multi-task LASSO | Adglob Infosystem Pvt Ltd