Getting Equivalent Results from glmnet and CVXR

Introduction

We’ve had several questions of the following type:

When I fit the same model in glmnet and CVXR, why are the results different?

For example, see this.

Obviously, unless one actually solves the same problem in both places, there’s no reason to expect the same result. The documentation for glmnet::glmnet clearly states the optimization objective and so one just has to ensure that the CVXR objective also matches that.

We illustrate below.

Lasso

Consider a simple Lasso fit from the glmnet example, for a fixed $\lambda$.

set.seed(123)
n <- 100; p <- 20; thresh <- 1e-12; lambda <- .05
x <-  matrix(rnorm(n * p), n, p); xDesign <- cbind(1, x)
y <-  rnorm(n)
fit1 <-  glmnet(x,y, lambda = lambda, thresh = thresh)

Warning in glmnet(x, y, lambda = lambda, thresh = thresh): Passing 'thresh' to
glmnet() is deprecated. Use control = list(thresh = ...) instead.

The glmnet documentation notes that the objective being maximized, in the default invocation, is

$$\frac{1}{2n}\parallel (y-X\beta ){\parallel }_2^2+\lambda \parallel {\beta }_{-1}{\parallel }_1,$$

where ${\beta }_{-1}$ is the beta vector excluding the first component, the intercept. Yes, the intercept is not penalized in the default invocation!

So we will use this objective with CVXR in the problem specification.

beta <- Variable(p + 1)
obj <- sum_squares(y - xDesign %*% beta) / (2 * n) + lambda * p_norm(beta[-1], 1)
prob <- Problem(Minimize(obj))
result <- psolve(prob, verbose = TRUE)

────────────────────────────────── CVXR v1.8.1 ─────────────────────────────────

ℹ Problem: 1 variable, 0 constraints (QP)

ℹ Compilation: "OSQP" via CVXR::Dcp2Cone -> CVXR::CvxAttr2Constr -> CVXR::ConeMatrixStuffing -> CVXR::OSQP_QP_Solver

ℹ Compile time: 0.152s

─────────────────────────────── Numerical solver ───────────────────────────────

-----------------------------------------------------------------
           OSQP v1.0.0  -  Operator Splitting QP Solver
              (c) The OSQP Developer Team
-----------------------------------------------------------------
problem:  variables n = 141, constraints m = 140
          nnz(P) + nnz(A) = 2380
settings: algebra = Built-in,
          OSQPInt = 4 bytes, OSQPFloat = 8 bytes,
          linear system solver = QDLDL v0.1.8,
          eps_abs = 1.0e-05, eps_rel = 1.0e-05,
          eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
          rho = 1.00e-01 (adaptive: 50 iterations),
          sigma = 1.00e-06, alpha = 1.60, max_iter = 10000
          check_termination: on (interval 25, duality gap: on),
          time_limit: 1.00e+10 sec,
          scaling: on (10 iterations), scaled_termination: off
          warm starting: on, polishing: on, 
iter   objective    prim res   dual res   gap        rel kkt    rho         time
   1  -8.0000e+00   8.00e+00   3.95e+01  -2.43e+02   3.95e+01   1.00e-01    1.85e-04s
  50   3.6435e-01   3.10e-02   9.80e-06  -4.85e-03   3.10e-02   9.06e-01*   4.33e-04s
 125   3.7110e-01   8.16e-06   2.07e-08  -1.16e-06   8.16e-06   9.06e-01    8.28e-04s
plsh   3.7110e-01   5.55e-16   1.02e-16   8.88e-17   5.55e-16   --------    9.74e-04s

status:               solved
solution polishing:   successful
number of iterations: 125
optimal objective:    0.3711
dual objective:       0.3711
duality gap:          8.8818e-17
primal-dual integral: 2.4302e+02
run time:             9.74e-04s
optimal rho estimate: 2.90e+00

──────────────────────────────────── Summary ───────────────────────────────────

✔ Status: optimal

✔ Optimal value: 0.371103

ℹ Compile time: 0.152s

ℹ Solver time: 0.011s

check_solver_status(prob)

We can print the coefficients side-by-side from glmnet and CVXR to compare. The results below should be close, and any differences are minor, due to different solver implementations.

est.table <- data.frame("CVXR.est" = value(beta), "GLMNET.est" = as.vector(coef(fit1)))
rownames(est.table) <- paste0("\\(\\beta_{", 0:p, "}\\)")
knitr::kable(est.table, format = "html", escape = FALSE, digits = 3) |>
    kable_styling("striped") |>
    column_spec(1:3, background = "#ececec")

	CVXR.est	GLMNET.est
${\beta }_0$	-0.125	-0.126
${\beta }_1$	-0.022	-0.028
${\beta }_2$	0.000	-0.002
${\beta }_3$	0.101	0.104
${\beta }_4$	0.000	0.000
${\beta }_5$	0.000	0.000
${\beta }_6$	0.000	0.000
${\beta }_7$	0.000	0.000
${\beta }_8$	-0.094	-0.091
${\beta }_9$	0.000	0.000
${\beta }_{10}$	0.000	0.000
${\beta }_{11}$	0.106	0.105
${\beta }_{12}$	0.000	0.000
${\beta }_{13}$	-0.057	-0.063
${\beta }_{14}$	0.000	0.000
${\beta }_{15}$	0.000	0.000
${\beta }_{16}$	0.000	0.000
${\beta }_{17}$	0.000	0.000
${\beta }_{18}$	0.000	0.000
${\beta }_{19}$	0.000	0.000
${\beta }_{20}$	-0.087	-0.083

A Penalized Logistic Example

We now consider a logistic fit, again with a penalized term with a specified $\lambda$.

lambda <- .025
y2 <- sample(x = c(0, 1), size = n, replace = TRUE)
fit2 <-  glmnet(x, y2, lambda = lambda, thresh = thresh, family = "binomial")

Warning in glmnet(x, y2, lambda = lambda, thresh = thresh, family =
"binomial"): Passing 'thresh' to glmnet() is deprecated. Use control =
list(thresh = ...) instead.

For logistic regression, the glmnet documentation states that the objective minimized is the negative log-likelihood divided by $n$ plus the penalty term which once again excludes the intercept in the default invocation. Below is the CVXR formulation, where we use the logistic atom as noted earlier in our other example on logistic regression.

beta <- Variable(p + 1)
obj2 <- (sum(xDesign[y2 <= 0, ] %*% beta) + sum(logistic(-xDesign %*% beta))) / n +
    lambda * p_norm(beta[-1], 1)
prob <- Problem(Minimize(obj2))
result <- psolve(prob)
check_solver_status(prob)

Once again, the results below should be close enough.

est.table <- data.frame("CVXR.est" = value(beta), "GLMNET.est" = as.vector(coef(fit2)))
rownames(est.table) <- paste0("\\(\\beta_{", 0:p, "}\\)")
knitr::kable(est.table, format = "html", escape = FALSE, digits = 3) |>
    kable_styling("striped") |>
    column_spec(1:3, background = "#ececec")

	CVXR.est	GLMNET.est
${\beta }_0$	-0.241	-0.243
${\beta }_1$	0.023	0.038
${\beta }_2$	-0.206	-0.204
${\beta }_3$	0.108	0.120
${\beta }_4$	0.000	0.000
${\beta }_5$	-0.303	-0.300
${\beta }_6$	-0.123	-0.131
${\beta }_7$	0.000	0.000
${\beta }_8$	-0.082	-0.082
${\beta }_9$	-0.056	-0.051
${\beta }_{10}$	0.011	0.014
${\beta }_{11}$	0.321	0.312
${\beta }_{12}$	0.000	0.000
${\beta }_{13}$	0.000	0.000
${\beta }_{14}$	0.000	0.000
${\beta }_{15}$	0.000	0.000
${\beta }_{16}$	0.000	0.000
${\beta }_{17}$	-0.214	-0.201
${\beta }_{18}$	-0.503	-0.502
${\beta }_{19}$	0.078	0.070
${\beta }_{20}$	0.219	0.211

Session Info

R version 4.5.2 (2025-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Tahoe 26.3

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] glmnet_4.2-9     Matrix_1.7-4     kableExtra_1.4.0 CVXR_1.8.1      

loaded via a namespace (and not attached):
 [1] gmp_0.7-5.1        clarabel_0.11.2    xml2_1.5.2         slam_0.1-55       
 [5] shape_1.4.6.1      stringi_1.8.7      lattice_0.22-9     digest_0.6.39     
 [9] magrittr_2.0.4     evaluate_1.0.5     grid_4.5.2         RColorBrewer_1.1-3
[13] iterators_1.0.14   fastmap_1.2.0      rprojroot_2.1.1    foreach_1.5.2     
[17] jsonlite_2.0.0     ECOSolveR_0.6.1    backports_1.5.0    survival_3.8-6    
[21] scs_3.2.7          Rmosek_11.1.1      viridisLite_0.4.3  scales_1.4.0      
[25] codetools_0.2-20   textshaping_1.0.4  cli_3.6.5          rlang_1.1.7       
[29] splines_4.5.2      Rglpk_0.6-5.1      yaml_2.3.12        otel_0.2.0        
[33] tools_4.5.2        osqp_1.0.0         Rcplex_0.3-8       checkmate_2.3.4   
[37] here_1.0.2         gurobi_13.0-1      vctrs_0.7.1        R6_2.6.1          
[41] lifecycle_1.0.5    stringr_1.6.0      htmlwidgets_1.6.4  cccp_0.3-3        
[45] glue_1.8.0         Rcpp_1.1.1         systemfonts_1.3.1  xfun_0.56         
[49] rstudioapi_0.18.0  knitr_1.51         dichromat_2.0-0.1  highs_1.12.0-3    
[53] farver_2.1.2       htmltools_0.5.9    rmarkdown_2.30     svglite_2.2.2     
[57] piqp_0.6.2         compiler_4.5.2     S7_0.2.1