glmnet and CVXRAnqi Fu and Balasubramanian Narasimhan
We’ve had several questions of the following type:
When I fit the same model in glmnet and CVXR, why are the results different?
For example, see this.
Obviously, unless one actually solves the same problem in both places, there’s no reason to expect the same result. The documentation for glmnet::glmnet clearly states the optimization objective and so one just has to ensure that the CVXR objective also matches that.
We illustrate below.
Consider a simple Lasso fit from the glmnet example, for a fixed \(\lambda\).
Warning in glmnet(x, y, lambda = lambda, thresh = thresh): Passing 'thresh' to
glmnet() is deprecated. Use control = list(thresh = ...) instead.The glmnet documentation notes that the objective being maximized, in the default invocation, is
\[ \frac{1}{2n}\|(y - X\beta)\|_2^2 + \lambda \|\beta_{-1}\|_1, \]
where \(\beta_{-1}\) is the beta vector excluding the first component, the intercept. Yes, the intercept is not penalized in the default invocation!
So we will use this objective with CVXR in the problem specification.
────────────────────────────────── CVXR v1.8.1 ─────────────────────────────────ℹ Problem: 1 variable, 0 constraints (QP)ℹ Compilation: "OSQP" via CVXR::Dcp2Cone -> CVXR::CvxAttr2Constr -> CVXR::ConeMatrixStuffing -> CVXR::OSQP_QP_Solverℹ Compile time: 0.105s─────────────────────────────── Numerical solver ───────────────────────────────-----------------------------------------------------------------
OSQP v1.0.0 - Operator Splitting QP Solver
(c) The OSQP Developer Team
-----------------------------------------------------------------
problem: variables n = 141, constraints m = 140
nnz(P) + nnz(A) = 2380
settings: algebra = Built-in,
OSQPInt = 4 bytes, OSQPFloat = 8 bytes,
linear system solver = QDLDL v0.1.8,
eps_abs = 1.0e-05, eps_rel = 1.0e-05,
eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
rho = 1.00e-01 (adaptive: 50 iterations),
sigma = 1.00e-06, alpha = 1.60, max_iter = 10000
check_termination: on (interval 25, duality gap: on),
time_limit: 1.00e+10 sec,
scaling: on (10 iterations), scaled_termination: off
warm starting: on, polishing: on,
iter objective prim res dual res gap rel kkt rho time
1 -8.0000e+00 8.00e+00 3.95e+01 -2.43e+02 3.95e+01 1.00e-01 1.20e-04s
50 3.6435e-01 3.10e-02 9.80e-06 -4.85e-03 3.10e-02 9.06e-01* 2.85e-04s
125 3.7110e-01 8.16e-06 2.07e-08 -1.16e-06 8.16e-06 9.06e-01 5.55e-04s
plsh 3.7110e-01 5.55e-16 1.02e-16 8.88e-17 5.55e-16 -------- 6.58e-04s
status: solved
solution polishing: successful
number of iterations: 125
optimal objective: 0.3711
dual objective: 0.3711
duality gap: 8.8818e-17
primal-dual integral: 2.4302e+02
run time: 6.58e-04s
optimal rho estimate: 2.90e+00──────────────────────────────────── Summary ───────────────────────────────────✔ Status: optimal✔ Optimal value: 0.371103ℹ Compile time: 0.105sℹ Solver time: 0.007sWe can print the coefficients side-by-side from glmnet and CVXR to compare. The results below should be close, and any differences are minor, due to different solver implementations.
| CVXR.est | GLMNET.est | |
|---|---|---|
| \(\beta_{0}\) | -0.125 | -0.126 |
| \(\beta_{1}\) | -0.022 | -0.028 |
| \(\beta_{2}\) | 0.000 | -0.002 |
| \(\beta_{3}\) | 0.101 | 0.104 |
| \(\beta_{4}\) | 0.000 | 0.000 |
| \(\beta_{5}\) | 0.000 | 0.000 |
| \(\beta_{6}\) | 0.000 | 0.000 |
| \(\beta_{7}\) | 0.000 | 0.000 |
| \(\beta_{8}\) | -0.094 | -0.091 |
| \(\beta_{9}\) | 0.000 | 0.000 |
| \(\beta_{10}\) | 0.000 | 0.000 |
| \(\beta_{11}\) | 0.106 | 0.105 |
| \(\beta_{12}\) | 0.000 | 0.000 |
| \(\beta_{13}\) | -0.057 | -0.063 |
| \(\beta_{14}\) | 0.000 | 0.000 |
| \(\beta_{15}\) | 0.000 | 0.000 |
| \(\beta_{16}\) | 0.000 | 0.000 |
| \(\beta_{17}\) | 0.000 | 0.000 |
| \(\beta_{18}\) | 0.000 | 0.000 |
| \(\beta_{19}\) | 0.000 | 0.000 |
| \(\beta_{20}\) | -0.087 | -0.083 |
We now consider a logistic fit, again with a penalized term with a specified \(\lambda\).
Warning in glmnet(x, y2, lambda = lambda, thresh = thresh, family =
"binomial"): Passing 'thresh' to glmnet() is deprecated. Use control =
list(thresh = ...) instead.For logistic regression, the glmnet documentation states that the objective minimized is the negative log-likelihood divided by \(n\) plus the penalty term which once again excludes the intercept in the default invocation. Below is the CVXR formulation, where we use the logistic atom as noted earlier in our other example on logistic regression.
Once again, the results below should be close enough.
| CVXR.est | GLMNET.est | |
|---|---|---|
| \(\beta_{0}\) | -0.241 | -0.243 |
| \(\beta_{1}\) | 0.023 | 0.038 |
| \(\beta_{2}\) | -0.206 | -0.204 |
| \(\beta_{3}\) | 0.108 | 0.120 |
| \(\beta_{4}\) | 0.000 | 0.000 |
| \(\beta_{5}\) | -0.303 | -0.300 |
| \(\beta_{6}\) | -0.123 | -0.131 |
| \(\beta_{7}\) | 0.000 | 0.000 |
| \(\beta_{8}\) | -0.082 | -0.082 |
| \(\beta_{9}\) | -0.056 | -0.051 |
| \(\beta_{10}\) | 0.011 | 0.014 |
| \(\beta_{11}\) | 0.321 | 0.312 |
| \(\beta_{12}\) | 0.000 | 0.000 |
| \(\beta_{13}\) | 0.000 | 0.000 |
| \(\beta_{14}\) | 0.000 | 0.000 |
| \(\beta_{15}\) | 0.000 | 0.000 |
| \(\beta_{16}\) | 0.000 | 0.000 |
| \(\beta_{17}\) | -0.214 | -0.201 |
| \(\beta_{18}\) | -0.503 | -0.502 |
| \(\beta_{19}\) | 0.078 | 0.070 |
| \(\beta_{20}\) | 0.219 | 0.211 |
R version 4.5.3 (2026-03-11)
Platform: aarch64-apple-darwin20
Running under: macOS Tahoe 26.3.1
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
time zone: America/Los_Angeles
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] glmnet_4.2-12 Matrix_1.7-4 kableExtra_1.4.0 CVXR_1.8.1
loaded via a namespace (and not attached):
[1] gmp_0.7-5.1 clarabel_0.11.2 xml2_1.5.2 slam_0.1-55
[5] shape_1.4.6.1 stringi_1.8.7 lattice_0.22-9 digest_0.6.39
[9] magrittr_2.0.4 evaluate_1.0.5 grid_4.5.3 RColorBrewer_1.1-3
[13] iterators_1.0.14 fastmap_1.2.0 rprojroot_2.1.1 foreach_1.5.2
[17] jsonlite_2.0.0 ECOSolveR_0.6.1 backports_1.5.0 survival_3.8-6
[21] scs_3.2.7 Rmosek_11.1.1 viridisLite_0.4.3 scales_1.4.0
[25] codetools_0.2-20 textshaping_1.0.5 cli_3.6.5 rlang_1.1.7
[29] splines_4.5.3 Rglpk_0.6-5.1 yaml_2.3.12 otel_0.2.0
[33] tools_4.5.3 osqp_1.0.0 checkmate_2.3.4 here_1.0.2
[37] gurobi_13.0-1 vctrs_0.7.1 R6_2.6.1 lifecycle_1.0.5
[41] stringr_1.6.0 htmlwidgets_1.6.4 cccp_0.3-3 glue_1.8.0
[45] Rcpp_1.1.1 systemfonts_1.3.2 xfun_0.56 rstudioapi_0.18.0
[49] knitr_1.51 dichromat_2.0-0.1 highs_1.12.0-3 farver_2.1.2
[53] htmltools_0.5.9 rmarkdown_2.30 svglite_2.2.2 piqp_0.6.2
[57] compiler_4.5.3 S7_0.2.1