Blog

Multi Task Elastic Net

It is an Elastic-Net model that allows to fit multiple regression problems jointly enforcing the selected features to be same for all the regression problems, also called tasks. Sklearn provides a linear model named MultiTaskElasticNet, trained with a mixed L1, L2-norm and L2 for regularisation, which estimates sparse coefficients for multiple regression problems jointly. In this, the response y is a 2D array of shape (n_samples, n_tasks).

Following is the objective function to minimize −minW12nsamples∥XW−Y∥2fro+αρ∥W∥21+α⟮1−ρ⟯2 ∥W∥2frominW12nsamples‖XW−Y‖fro2+αρ‖W‖21+α⟮1−ρ⟯2 ‖W‖fro2

As in MultiTaskLasso, here also, Fro indicates the Frobenius norm −∥A∥Fro=∑ij−−−√a2ij

And L1L2 leads to the following −∥A∥21=∑i∑j−−−√a2ij

The parameters and the attributes for MultiTaskElasticNet are like that of ElasticNet. The only difference is in li_ratio i.e. ElasticNet mixing parameter. In MultiTaskElasticNet its range is 0 < l1_ratio < = 1. If l1_ratio = 1, the penalty would be L1/L2 penalty. If l1_ratio = 0, the penalty would be an L2 penalty. If the value of l1 ratio is between 0 and 1, the penalty would be the combination of L1/L2 and L2.

And, opposite to ElasticNet, MultiTaskElasticNet doesn’t have precompute attribute.

Implementation Example

To show the difference, we are implementing the same example as we did in Multi-task Lasso −

from sklearn import linear_model
MTENReg = linear_model.MultiTaskElasticNet(alpha = 0.5)
MTENReg.fit([[0,0], [1, 1], [2, 2]], [[0, 0],[1,1],[2,2]])

Output

MultiTaskElasticNet(alpha = 0.5, copy_X = True, fit_intercept = True, l1_ratio = 0.5,
max_iter = 1000, normalize = False, random_state = None,
selection = 'cyclic', tol = 0.0001, warm_start = False)

Example

#Predicting new values
MTENReg.predict([[1,0]])

Output

array([[0.69056563, 0.69056563]])

Example

#weight vectors
MTENReg.coef_

Output

array([[0.30943437, 0.30938224],
[0.30943437, 0.30938224]])

Example

#Calculating intercept
MTENReg.intercept_

Output

array([0.38118338, 0.38118338])

Example

#Calculating number of iterations
MTENReg.n_iter_

Output

15

Next Topic : Click Here