CONOPT4 and MOSEK option files

I’m running a model all in conic forms. My colleagues using CONOPT4 (and the options below) are getting sensible results (calibration and appropriate scenarios responses). But as I switch solvers to MOSEK (using exactly the same model), results are not looking good at all ( no scenarios responses, e. g.).
Thus, which options should I use in MOSEK that are equivalent to those below?


  • Option file for Solving Models

  • Maximum and minimum jacobian element
    rtmaxj = 1.00e+20
  • internal value of infinity
    rtmaxv = 1.00e+30
  • scaling
    lsscal = t
    lfscal = 20
  • limit for slow progress
    *lfnicr = 3000
    lfnicr = 4000
  • limit for superbasics
    lfnsup = 10000
  • iteration log frequency
    lfilog = 100
  • iteration log frequency for SLP iterations
    lfilos = 100

rtnwtr = 1.00e-5
rtnwmi = 1.00e-8
*rtnwmi = 2.50e-5

  • increases the number of stalled iterations
    *lfstal = 200
    lfstal = 500
    *workfactor=2

Fermar,

You did not include your model, or any sort of GAMS code to use to reproduce your results, so it’s difficult to say exactly what is needed. Looking at your CONOPT options, the most striking thing to me is that you have

* Maximum and minimum jacobian element
rtmaxj = 1.00e+20
* internal value of infinity
rtmaxv = 1.00e+30

Probably you set these because CONOPT suggested you do so, which it would do if your model was very poorly scaled. In such a case, you would be better off asking how you can properly scale your model so that these options are not needed. You can read about this here:

https://www.gams.com/41/docs/UG_NLP_GoodFormulations.html#UG_NLP_GoodFormulations_Scaling
https://www.gams.com/41/docs/S_CONOPT.html?search=scaling#CONOPT_SCALING

It’s tempting to think or hope that different solvers will always produce the same results when given the same inputs, but this is simply not the case. It often happens that we reach this happy situation - aided by a convex model, good scaling, a good initial point, etc. - but it is not to be assumed.

HTH,

-Steve

The question is cross-posted at

https://groups.google.com/g/mosek/c/-H4QZptSo80

(My best guess is that the conversion of the model to conic form is not correct.)

Erling

Hi, thanks for the reply. I attach the MOSEK specification only as I don’t have the CONOPT license, it was run in a different computer.

On line 244 is the start of the first stage of positive math programming to get marginals for calibration constraints. On line 395 is the start of the second stage to construct the exponential cost curve.

On line 523 is the third stage where the exponential cost curve is entered in the OF, I calibrate for the number of cows. And it calibrates.

The problems start on line 716 for the policy model where I enter the sim_price_methane which is a proxy for emissions pricing (methane). The number of cows variable does not react at all, if I increase the price the reactions are wild for some units but not for others. After extensive exploration of scaling problems, I can’t find the source .

I appreciate every feedback

M



PS: the gdx file is for 10 units only, the original file is more than 10 thousand
NOM_16112022_gf.gms (37.5 KB)
NOM_Input_v2_gf.gdx (9.88 KB)

I posted a reply in the Mosek Google Group thread.

Erling, here are the log and task files. One is with explicit user scaling while the other one is without user scaling. -Michael
scaled.log (7.57 KB)
unscaled.log (8.96 KB)
tasks.zip (12.9 KB)

The problem you have is kind of nasty and v10.0 struggles with it. It does not solve your problem. One thing I can see is that the norm of the primal solution is quite large i.e. 2e7 and that is bad.

However, the current development version of Mosek solves the problem quite OK as you can see below. Most likely it is some additional stuff in the presolve that makes your problem solvable.
I am looking into that and maybe that can tell us what it is in your problem that makes it hard.

26 8.8e-11 1.6e-14 6.4e-17 1.00e+00 1.025095051e+07 1.025095051e+07 1.5e-16 0.03
Optimizer terminated. Time: 0.03

Open file ‘out\jobs\runmosek\stat\rue\default-scaled.stat’

Interior-point solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 1.0250950506e+07 nrm: 2e+07 Viol. con: 6e-11 var: 0e+00 cones: 0e+00
Dual. obj: 1.0250950505e+07 nrm: 8e+04 Viol. con: 0e+00 var: 1e-06 cones: 0e+00

Btw Mosek chooses to solve the dual problem which seems to cause issues in 10.0. If you force Mosek to solve the primal problem it seems you get a decent solution.

So you should set the parameter

MSK_IPAR_SIM_SOLVE_FORM

to force Mosek to solve the primal problem.

See

https://www.gams.com/latest/docs/S_MOSEK.html#MOSEK_MOSEK_OPTIONS

for details.

Thanks for that.
My MOsek version does solve the model. But the simulation in the fourth stage does not produce adequate responses.
Any hint on the major scaling issues coming from?


Erling,

MOSEK succeeds with the third model in fermar’s GAMS file when using MSK_IPAR_SIM_SOLVE_FORM MSK_SOLVE_PRIMAL. The fourth model still fails (I tried both primal and dual). I have attached log and task.

-Michael
FOurthStage.zip (8.65 KB)

Thanks Michael.

Yesterday, I managed to backport the stuff in presolve that made the models solvable.
You can see the result below which are quite good. We plan to release a new version 10.0 shortly.

The problems are somewhat numerically challenging. I observed the model seems to contain

maximize something -t

t >= e^x

in your model where 0<= x <=20. e^20 is a huge number. If that upper bound on x is not redundant, then that is a bit fishy.

Latest v10 results:

Interior-point solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 9.9750645079e+06 nrm: 2e+07 Viol. con: 6e-11 var: 0e+00 cones: 0e+00
Dual. obj: 9.9750645078e+06 nrm: 8e+04 Viol. con: 0e+00 var: 3e-06 cones: 0e+00

Optimizer summary
Optimizer - time: 0.03
Interior-point - iterations : 27 time: 0.03

Thanks Michael.

Yesterday, I managed to backport the stuff in presolve that made the models solvable.
You can see the result below which are quite good. We plan to release a new version 10.0 shortly.

Latest v10 results:

Interior-point solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 9.9750645079e+06 nrm: 2e+07 Viol. con: 6e-11 var: 0e+00 cones: 0e+00
Dual. obj: 9.9750645078e+06 nrm: 8e+04 Viol. con: 0e+00 var: 3e-06 cones: 0e+00

Optimizer summary
Optimizer - time: 0.03
Interior-point - iterations : 27 time: 0.03

We have released version 10.0.28 solve all 4 instances well.