Entropy Maximization

Author

CVXPY Developers and Balasubramanian Narasimhan

Introduction

Adapted from the CVX example of the same name, by Joelle Skaf, 4/24/2008.

Consider the linear inequality constrained entropy maximization problem:

$\begin{array}{ll} maximize & - \sum_{i = 1}^{n} x_{i} \log (x_{i}) \\ subject to & \sum_{i = 1}^{n} x_{i} = 1 \\ F x ⪯ g, \end{array}$

where the variable is $x \in R^{n}$ .

This problem can be formulated in CVXR using the entr atom, which computes the elementwise entropy $- x_{i} \log (x_{i})$ .

Generate Problem Data

## Make random input repeatable
set.seed(0)

## Matrix size parameters
n <- 20
m <- 10
p <- 5

## Generate random problem data
tmp <- runif(n)
A_mat <- matrix(rnorm(m * n), nrow = m, ncol = n)
b_vec <- A_mat %*% tmp
F_mat <- matrix(rnorm(p * n), nrow = p, ncol = n)
g_vec <- F_mat %*% tmp + runif(p)

Formulate and Solve

## Entropy maximization
x <- Variable(n)
obj <- Maximize(sum(entr(x)))
constraints <- list(
    A_mat %*% x == b_vec,
    F_mat %*% x <= g_vec
)
prob <- Problem(obj, constraints)
result <- psolve(prob, verbose = TRUE)

────────────────────────────────── CVXR v1.8.1 ─────────────────────────────────

ℹ Problem: 1 variable, 2 constraints (DCP)

ℹ Compilation: "CLARABEL" via CVXR::FlipObjective -> CVXR::Dcp2Cone -> CVXR::CvxAttr2Constr -> CVXR::ConeMatrixStuffing -> CVXR::Clarabel_Solver

ℹ Compile time: 0.202s

─────────────────────────────── Numerical solver ───────────────────────────────

──────────────────────────────────── Summary ───────────────────────────────────

✔ Status: optimal

✔ Optimal value: 6.17543

ℹ Compile time: 0.202s

ℹ Solver time: 0.073s

check_solver_status(prob)

## Print result
cat("\nThe optimal value is:", result, "\n")
cat("\nThe optimal solution is:\n")
print(round(value(x), 6))


The optimal value is: 6.17543 

The optimal solution is:
          [,1]
 [1,] 0.331911
 [2,] 0.489598
 [3,] 0.484109
 [4,] 0.507501
 [5,] 0.740520
 [6,] 0.255377
 [7,] 0.676289
 [8,] 0.826750
 [9,] 0.742041
[10,] 0.249216
[11,] 0.245718
[12,] 0.275945
[13,] 0.429542
[14,] 0.508571
[15,] 0.536493
[16,] 0.596709
[17,] 0.727919
[18,] 0.279349
[19,] 0.765691
[20,] 0.430987

Session Info

R version 4.5.2 (2025-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Tahoe 26.3

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] CVXR_1.8.1

loaded via a namespace (and not attached):
 [1] slam_0.1-55       cli_3.6.5         knitr_1.51        ECOSolveR_0.6.1  
 [5] rlang_1.1.7       xfun_0.56         clarabel_0.11.2   otel_0.2.0       
 [9] gurobi_13.0-1     Rglpk_0.6-5.1     highs_1.12.0-3    cccp_0.3-3       
[13] scs_3.2.7         S7_0.2.1          jsonlite_2.0.0    Rcplex_0.3-8     
[17] backports_1.5.0   rprojroot_2.1.1   htmltools_0.5.9   Rmosek_11.1.1    
[21] gmp_0.7-5.1       piqp_0.6.2        rmarkdown_2.30    grid_4.5.2       
[25] evaluate_1.0.5    fastmap_1.2.0     yaml_2.3.12       compiler_4.5.2   
[29] codetools_0.2-20  htmlwidgets_1.6.4 Rcpp_1.1.1        here_1.0.2       
[33] osqp_1.0.0        lattice_0.22-9    digest_0.6.39     checkmate_2.3.4  
[37] Matrix_1.7-4      tools_4.5.2

Introduction

Generate Problem Data

Formulate and Solve

Session Info

References