Andy:
The answer to your first question is that GAMS only allocates memory to the
nonzero values stored in a parameter.
The “option profile=1;” statement that you seem to be using will tell you
have many nonzeros there are associated with each statement. Using the
ramsey.gms model from the GAMS Library you get something like:
---- 67 Assignment beta 0.000 0.000 SECS 4 Mb 11
---- 68 Assignment beta 0.000 0.000 SECS 4 Mb 11
---- 77 Assignment a 0.000 0.000 SECS 4 Mb 0
---- 78 Assignment al 0.000 0.000 SECS 4 Mb 11
where the last number is the number of nonzeros in the parameter after the
assignment. For the model generation part the numbers are like
---- 138 Solve Init ramsey 0.000 0.000 SECS 6 Mb
---- 102 Equation cc 0.000 0.000 SECS 6 Mb 11
---- 104 Equation kk 0.000 0.000 SECS 6 Mb 10
---- 106 Equation tc 0.000 0.000 SECS 6 Mb 1
---- 108 Equation util 0.000 0.000 SECS 6 Mb 1
and the last number is the number of equations in each block.
One type of statement that I have found sometimes uses a lot of memory is:
Var.fx(I,j,k) = 0;
The rule about only storing nonzero value does not apply for variables. Here
GAMS only store ‘non-default variable records’. For a variable GAMS stores
the four numbers var.lo, var.up, var.l, and var.m. If just one of these four
numbers is non-default then the whole record is stored. For a positive
variable the default is (0,+inf,0,0) and similarly for other variable types.
When you fix a variable it is bound to be non-default. Instead of fixing,
use $-conditions in the equations to avoid the variable you do not want.
Anyway, look for the statements where the amount of memory (the second to
last number in the profile output) increases a lot.
Good hunt
Arne
Arne Stolbjerg Drud
ARKI Consulting & Development A/S
Bagsvaerdvej 246A, DK-2880 Bagsvaerd, Denmark
Phone: (+45) 44 49 03 23, Fax: (+45) 44 49 03 33, email: adrud@arki.dk
-----Original Message-----
From: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] On
Behalf Of AC
Sent: Wednesday, March 28, 2012 12:50 AM
To: gamsworld
Subject: How Can I: identify memory hogs
Hi
We have a model that is starting to get large and we think this is causing
us execution time problems (12hrs before the solve begins).
The profile option shows us that right before the solve statement it was
over 12Gb. We would like to start trying to manage this and get the memory
usage down.
There are definitely some parameters we have which are over populated, we
calc values for them but never end up using them. As a made up example,
imagine we calculate a TransportCost(i,j) between all nodes i and j, yet
when we solve the model we only ever consider transporting between a subset
of nodes. In this case we could get smarter and only calculate a
TransportCost(i,j) for the valid transport routes.
The calculation time of the parameters does not seem to be the issue, we did
some profiling and the worst line of code was only 30secs. We suspect the
issue is memory usage and paging etc, so if we can crank the memory usage
down then this might help.
My question is:
-
could someone confirm that GAMS will use less memory if we more sparsely
populate our parameters. I guess I am asking if GAMS pre- allocates all the
memory it needs for a parameter, or whether it populates it on the fly.
-
is there any easy way to identify which parameters are major size/ memory
hogs? Being able to quickly see which parameters are the killers and then
figuring out if we are over populating them would be really helpful as we
could focus of the worst offenders first.
Thanks in advance
AndyC
–
“gamsworld” group.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.
\