% Section 7.2, Figures 7.2-7.3 % Boyd & Vandenberghe, "Convex Optimization" % Originally by Lieven Vandenberghe % Adapted for CVX by Michael Grant 4/11/06 % We consider a probability distribution on 100 equidistant points in the % interval [-1,1]. We impose the following prior assumptions: % -0.1 <= E(X) <= +0.1 % +0.5 <= E(X^2) <= +0.6 % -0.3 <= E(3*X^3-2*X) <= -0.2 % +0.3 <= Pr(X<0) <= 0.4 % Along with the constraints sum(p) == 1, p >= 0, these constraints % describe a polyhedron of probability distrubtions. In the first figure, % the distribution that maximizes entropy is computed. In the second % figure, we compute upper and lower bounds for Prob(X<=a_i) for each % point -1 <= a_i <= +1 in the distribution, as well as the maximum % entropy CDF. % Represent the polyhedron as follows: % A * p <= b % sum( p ) == 1 % p >= 0 n = 100; a = linspace(-1,1,n); a2 = a .^ 2; a3 = 3 * ( a.^ 3 ) - 2 * a; ap = +( a < 0 ); A = [ a ; -a ; a2 ; -a2 ; a3 ; -a3 ; ap ; -ap ]; b = [ 0.1 ; 0.1 ;0.5 ; -0.5 ; -0.2 ; 0.3 ; 0.4 ; -0.3 ]; % Compute the maximum entropy distribution cvx_expert true cvx_begin variables pent(n) maximize( sum(entr(pent)) ) sum(pent) == 1; A * pent <= b; cvx_end % Compute the bounds on Prob(X<=a_i), i=1,...,n Ubnds = zeros(1,n); Lbnds = zeros(1,n); for t = 1 : n, cvx_begin quiet variable p( n ) minimize sum( p(1:t) ) p >= 0; sum( p ) == 1; A * p <= b; cvx_end Lbnds(t) = cvx_optval; cvx_begin quiet variable p( n ) maximize sum( p(1:t) ) p >= 0; sum( p ) == 1; A * p <= b; cvx_end Ubnds(t) = cvx_optval; disp( sprintf( '%g <= Prob(x<=%g) <= %g' , Lbnds(t), a(t), Ubnds(t) ) ); % Generate the figures figure( 1 ) stairs( a, pent ); xlabel( 'x' ); ylabel( 'PDF( x )' ); figure( 2 ) stairs( a, cumsum( pent ) ); grid on ; hold on d = stairs(a, Lbnds, 'r-' ); set(d, 'Color' ,[0 0.5 0]); d = stairs(a, Ubnds, 'r-' ); set(d, 'Color' ,[0 0.5 0]); d = plot([-1,-1], [Lbnds(1), Ubnds(1)], 'r-' ); set(d, 'Color' ,[0 0.5 0]); axis([-1.1 1.1 -0.1 1.1]); xlabel( 'x' ); ylabel( 'CDF( x )' ); hold off Calling Mosek 9.1.9: 308 variables, 109 equality constraints ------------------------------------------------------------ MOSEK Version 9.1.9 (Build date: 2019-11-21 11:32:15) Platform: MACOSX/64-X86 Problem Name : Objective sense : min Type : CONIC (conic optimization problem) Constraints : 109 Cones : 100 Scalar variables : 308 Matrix variables : 0 Integer variables : 0 Optimizer started. Presolve started. Linear dependency checker started. Linear dependency checker terminated. Eliminator started. Freed constraints in eliminator : 0 Eliminator terminated. Eliminator - tries : 1 time : 0.00 Lin. dep. - tries : 1 time : 0.00 Lin. dep. - number : 0 Presolve terminated. Time: 0.00 Problem Name : Objective sense : min Type : CONIC (conic optimization problem) Constraints : 109 Cones : 100 Scalar variables : 308 Matrix variables : 0 Integer variables : 0 Optimizer - threads : 8 Optimizer - solved problem : the primal Optimizer - Constraints : 5 Optimizer - Cones : 100 Optimizer - Scalar variables : 303 conic : 300 Optimizer - Semi-definite variables: 0 scalarized : 0 Factor - setup time : 0.00 dense det. time : 0.00 Factor - ML order time : 0.00 GP order time : 0.00 Factor - nonzeros before factor : 15 after factor : 15 Factor - dense dim. : 0 flops : 5.06e+03 ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME 0 8.0e+01 8.1e-01 2.1e+02 0.00e+00 8.278383991e+01 -1.290927710e+02 1.0e+00 0.00 1 9.2e+00 9.3e-02 1.2e+01 6.35e-01 -1.258106333e+01 -4.378494190e+01 1.2e-01 0.01 2 1.1e+00 1.2e-02 8.5e-01 5.78e-01 -7.925074971e+00 -1.308368634e+01 1.4e-02 0.01 3 1.9e-01 1.9e-03 6.5e-02 7.63e-01 -4.920183301e+00 -5.898967691e+00 2.4e-03 0.01 4 1.9e-02 1.9e-04 1.6e-03 1.14e+00 -4.426324685e+00 -4.514412441e+00 2.4e-04 0.01 5 1.7e-03 1.7e-05 3.9e-05 1.05e+00 -4.389622164e+00 -4.397222024e+00 2.1e-05 0.01 6 9.8e-05 9.9e-07 5.4e-07 1.01e+00 -4.386539903e+00 -4.386981889e+00 1.2e-06 0.01 7 9.9e-06 1.0e-07 1.7e-08 1.00e+00 -4.386323068e+00 -4.386367836e+00 1.2e-07 0.01 8 1.3e-06 1.3e-08 8.4e-10 1.00e+00 -4.386298489e+00 -4.386304439e+00 1.7e-08 0.01 9 1.8e-07 1.8e-09 4.1e-11 1.00e+00 -4.386294908e+00 -4.386295698e+00 2.2e-09 0.01 10 5.3e-08 5.3e-10 6.7e-12 1.00e+00 -4.386294501e+00 -4.386294738e+00 6.6e-10 0.01 11 3.2e-08 3.2e-10 3.1e-12 9.99e-01 -4.386294437e+00 -4.386294581e+00 4.0e-10 0.01 12 3.2e-08 3.2e-10 3.1e-12 1.08e+00 -4.386294436e+00 -4.386294578e+00 4.0e-10 0.01 13 3.1e-08 3.2e-10 3.0e-12 9.85e-01 -4.386294435e+00 -4.386294575e+00 3.9e-10 0.02 14 3.1e-08 3.2e-10 3.0e-12 9.97e-01 -4.386294435e+00 -4.386294575e+00 3.9e-10 0.02 15 7.6e-09 7.9e-11 3.5e-13 9.99e-01 -4.386294359e+00 -4.386294394e+00 9.7e-11 0.02 Optimizer terminated. Time: 0.02 Interior-point solution summary Problem status : PRIMAL_AND_DUAL_FEASIBLE Solution status : OPTIMAL Primal. obj: -4.3862943588e+00 nrm: 1e+00 Viol. con: 1e-08 var: 0e+00 cones: 0e+00 Dual. obj: -4.3862943940e+00 nrm: 5e+00 Viol. con: 0e+00 var: 6e-13 cones: 0e+00 Optimizer summary Optimizer - time: 0.02 Interior-point - iterations : 15 time: 0.02 Basis identification - time: 0.00 Primal - iterations : 0 time: 0.00 Dual - iterations : 0 time: 0.00 Clean primal - iterations : 0 time: 0.00 Clean dual - iterations : 0 time: 0.00 Simplex - time: 0.00 Primal simplex - iterations : 0 time: 0.00 Dual simplex - iterations : 0 time: 0.00 Mixed integer - relaxations: 0 time: 0.00 ------------------------------------------------------------ Status: Solved Optimal value (cvx_optval): +4.38629 0 <= Prob(x<=-1) <= 0.329406 0 <= Prob(x<=-0.979798) <= 0.344777 0 <= Prob(x<=-0.959596) <= 0.360771 0 <= Prob(x<=-0.939394) <= 0.377365 0 <= Prob(x<=-0.919192) <= 0.394532 0 <= Prob(x<=-0.89899) <= 0.4 0 <= Prob(x<=-0.878788) <= 0.4 0 <= Prob(x<=-0.858586) <= 0.4 0 <= Prob(x<=-0.838384) <= 0.4 0 <= Prob(x<=-0.818182) <= 0.4 0 <= Prob(x<=-0.79798) <= 0.4 0.0116304 <= Prob(x<=-0.777778) <= 0.4 0.0331953 <= Prob(x<=-0.757576) <= 0.4 0.0519157 <= Prob(x<=-0.737374) <= 0.4 0.0701091 <= Prob(x<=-0.717172) <= 0.4 0.0859495 <= Prob(x<=-0.69697) <= 0.4 0.0998612 <= Prob(x<=-0.676768) <= 0.4 0.112141 <= Prob(x<=-0.656566) <= 0.4 0.123045 <= Prob(x<=-0.636364) <= 0.4 0.132778 <= Prob(x<=-0.616162) <= 0.4 0.141527 <= Prob(x<=-0.59596) <= 0.4 0.149418 <= Prob(x<=-0.575758) <= 0.4 0.15655 <= Prob(x<=-0.555556) <= 0.4 0.163015 <= Prob(x<=-0.535354) <= 0.4 0.168895 <= Prob(x<=-0.515152) <= 0.4 0.174283 <= Prob(x<=-0.494949) <= 0.4 0.179205 <= Prob(x<=-0.474747) <= 0.4 0.18371 <= Prob(x<=-0.454545) <= 0.4 0.187841 <= Prob(x<=-0.434343) <= 0.4 0.191651 <= Prob(x<=-0.414141) <= 0.4 0.195164 <= Prob(x<=-0.393939) <= 0.4 0.198396 <= Prob(x<=-0.373737) <= 0.4 0.201373 <= Prob(x<=-0.353535) <= 0.4 0.204127 <= Prob(x<=-0.333333) <= 0.4 0.206681 <= Prob(x<=-0.313131) <= 0.4 0.209037 <= Prob(x<=-0.292929) <= 0.4 0.211209 <= Prob(x<=-0.272727) <= 0.4 0.213219 <= Prob(x<=-0.252525) <= 0.4 0.215088 <= Prob(x<=-0.232323) <= 0.4 0.216811 <= Prob(x<=-0.212121) <= 0.4 0.218398 <= Prob(x<=-0.191919) <= 0.4 0.219862 <= Prob(x<=-0.171717) <= 0.4 0.221224 <= Prob(x<=-0.151515) <= 0.4 0.222474 <= Prob(x<=-0.131313) <= 0.4 0.223619 <= Prob(x<=-0.111111) <= 0.4 0.224669 <= Prob(x<=-0.0909091) <= 0.4 0.225643 <= Prob(x<=-0.0707071) <= 0.4 0.22653 <= Prob(x<=-0.0505051) <= 0.4 0.227334 <= Prob(x<=-0.030303) <= 0.4 0.3 <= Prob(x<=-0.010101) <= 0.4 0.3 <= Prob(x<=0.010101) <= 0.778942 0.3 <= Prob(x<=0.030303) <= 0.792532 0.3 <= Prob(x<=0.0505051) <= 0.806483 0.3 <= Prob(x<=0.0707071) <= 0.819022 0.3 <= Prob(x<=0.0909091) <= 0.825 0.3 <= Prob(x<=0.111111) <= 0.83125 0.3 <= Prob(x<=0.131313) <= 0.837791 0.3 <= Prob(x<=0.151515) <= 0.841937 0.3 <= Prob(x<=0.171717) <= 0.845957 0.3 <= Prob(x<=0.191919) <= 0.850137 0.3 <= Prob(x<=0.212121) <= 0.854492 0.3 <= Prob(x<=0.232323) <= 0.859052 0.3 <= Prob(x<=0.252525) <= 0.863811 0.3 <= Prob(x<=0.272727) <= 0.868817 0.3 <= Prob(x<=0.292929) <= 0.874066 0.3 <= Prob(x<=0.313131) <= 0.877055 0.3 <= Prob(x<=0.333333) <= 0.880067 0.3 <= Prob(x<=0.353535) <= 0.883272 0.300787 <= Prob(x<=0.373737) <= 0.886687 0.307695 <= Prob(x<=0.393939) <= 0.890333 0.314397 <= Prob(x<=0.414141) <= 0.894234 0.320909 <= Prob(x<=0.434343) <= 0.898418 0.327232 <= Prob(x<=0.454545) <= 0.902981 0.333379 <= Prob(x<=0.474747) <= 0.909013 0.339323 <= Prob(x<=0.494949) <= 0.916606 0.345134 <= Prob(x<=0.515152) <= 0.925292 0.350719 <= Prob(x<=0.535354) <= 0.935184 0.356201 <= Prob(x<=0.555556) <= 0.946304 0.361491 <= Prob(x<=0.575758) <= 0.958921 0.366603 <= Prob(x<=0.59596) <= 0.973265 0.371622 <= Prob(x<=0.616162) <= 0.989508 0.387329 <= Prob(x<=0.636364) <= 1 0.410495 <= Prob(x<=0.656566) <= 1 0.439031 <= Prob(x<=0.676768) <= 1 0.466372 <= Prob(x<=0.69697) <= 1 0.492663 <= Prob(x<=0.717172) <= 1 0.518025 <= Prob(x<=0.737374) <= 1 0.542592 <= Prob(x<=0.757576) <= 1 0.56651 <= Prob(x<=0.777778) <= 1 0.589941 <= Prob(x<=0.79798) <= 1 0.613125 <= Prob(x<=0.818182) <= 1 0.635881 <= Prob(x<=0.838384) <= 1 0.657609 <= Prob(x<=0.858586) <= 1 0.678314 <= Prob(x<=0.878788) <= 1 0.697846 <= Prob(x<=0.89899) <= 1 0.716238 <= Prob(x<=0.919192) <= 1 0.733536 <= Prob(x<=0.939394) <= 1 0.74974 <= Prob(x<=0.959596) <= 1 0.764914 <= Prob(x<=0.979798) <= 1 1 <= Prob(x<=1) <= 1