Grid Computing

GAMS
1

Hi,

I’m trying to follow McCarl’s user guide to implement grid computing
on my 8 processor server, but not quite able.

I have a solution loop that performs a home-made multi-start for a
nonlinear problem. I’m not using KNITRO’s multistart heuristic
because I need to analyze the data for each run, and I can’t get
access to the runs in KNITRO. (Also, I don’t think I can multithread
the runs in KNITRO, as I’m trying to do here).

So I have a set xx with values xx1 to xx100000. I typically do a loop
over xx where, within each loop, I set the variables to values drawn
from a uniform distribution of the feasible values. I then store some
variables in a parameter RunData. My solve-related code looks
something like the following (although with lots of other goodies):

Loop(xx,
{bunch of code setting random initial guesses}
Solve MyModel using NLP maximizing MarketValue;
RunData(xx,‘Obj’) = MarketValue.l;
RunData(xx,‘Infes’) = MyModel.numinfes;
RunData(xx,‘MyModelstat’) = MyModel.modelstat;
RunData(xx,‘MySolvestat’) = MyModel.solvestat;
RunData(xx,‘MyTime’) = MyModel.resusd;
RunData(xx,‘Pgen1’) = PGen.l(‘g00001’);
{bunch of other code capturing whatever vars I’m interested
in}
yy = yy + 1;
if (round(yy/100) = yy/100,
execute_unload ‘ParamTestOutput’;
);
if (ACOPF.modelstat = 2,
execute_unload ‘Success’;
);
);

It should be pretty obvious what this does. At the end, I can open
ParamTestOutput and scroll through the RunData table (or copy it to
Excel to do some analysis about different solutions, run times, etc.).

I do a LOT of these, so I want to use all 8 of my processors to speed
things up, if I can. I think I understand that I can do that using
solvestat = 3. As I said, I reviewed the McCarl guide, but can’t
quite figure out what’s going on there. Any help would be great.

I now have the " if (round(yy/100) = yy/100…" code in there so that
it stores the results every 100 runs (so that I can see what’s going
on without waiting for thousands or hundreds of thousands of runs to
complete). That’s not necessary, but if I could have some feature
like that in my grid-enabled code, it would really help. Or do I just
n

By the way, I’ve asked a number of other researchers who use GAMS
pretty regularly, and they were all very interested in this issue, but
weren’t able to understand the user guide here either. So you’ll be
helping several people, as they wanted me to forward the answer to
them as soon as I have it.

Thanks much!
Emily


\

2

Thanks much. I’ll look at your files carefully today.

I am wondering if my situation is slightly different from yours
though. I think I actually do need to run only the solve step in
parallel. After all, I am using the same model, just with different
initual guesses for different solve runs.

If I run my model with solvestat=3, I do get a huge speed-up (I
haven’t clocked it, but it appears to be roughly equal to the number
of processors I am using). My problem is that, when the runs are
completed, I don’t know where the data went. It isn’t stored in the
parameters I defined for storing my data. McCarl describes the method
for collecting the data, but I can’t quite follow it.

Thanks,
Emily

On Dec 10, 7:51 am, “Arne Stolbjerg Drud” wrote:

Hi Emily

I had a similar problem some time ago and found that grid computing was not
the right tool for me. With grid computing, all models are generated
sequentially and the collection of the solutions is also done sequentially.
Only the solve step is in parallel and for my case this was very little and
the overall speedup was very small.

But you can instead create n=2, 4, or 8 jobs with 100000/n solves in each
job, run them in parallel, and then collect the data. I have attached the
files for a small example based on the old trnsport model. The basic model
is in model.gms and the interesting things are at the end. models.gms has
three command-line arguments, --from=, --to=, and --thread=. The first two
defines the looping set and the last is used to name individual gdx-files
with the relevant solution information.

The second file is driver.gms that controls the calls of model.gms with
various arguments. It is done by creating a bat-file (s.bat – Windows only)
that is executed. The command ‘start gams …’ will run each gams program in
a separate window and they will run in parallel. After the models have
finished the data in the gdx-files are collected into one dataset using the
doload.gms bat-include file. You can run this in a separate program – I
have put it at the end of driver.gms and since this program continues to
execute while the individual models are being solved I have to wait until
the gdx-files are available before reading them – therefore all the test
and sleep business.

Hope you can use the ideas.

Arne

Arne Stolbjerg Drud
ARKI Consulting & Development A/S
Bagsvaerdvej 246A, DK-2880 Bagsvaerd, Denmark
Phone: (+45) 44 49 03 23, Fax: (+45) 44 49 03 33, email: ad…@arki.dk

-----Oprindelig meddelelse-----
Fra: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] PÃ¥ vegne
af emily
Sendt: 10. december 2010 01:29
Til: gamsworld
Emne: Grid Computing

Hi,

I’m trying to follow McCarl’s user guide to implement grid computing
on my 8 processor server, but not quite able.

I have a solution loop that performs a home-made multi-start for a
nonlinear problem. I’m not using KNITRO’s multistart heuristic
because I need to analyze the data for each run, and I can’t get
access to the runs in KNITRO. (Also, I don’t think I can multithread
the runs in KNITRO, as I’m trying to do here).

So I have a set xx with values xx1 to xx100000. I typically do a loop
over xx where, within each loop, I set the variables to values drawn
from a uniform distribution of the feasible values. I then store some
variables in a parameter RunData. My solve-related code looks
something like the following (although with lots of other goodies):

Loop(xx,
{bunch of code setting random initial guesses}
Solve MyModel using NLP maximizing MarketValue;
RunData(xx,‘Obj’) = MarketValue.l;
RunData(xx,‘Infes’) = MyModel.numinfes;
RunData(xx,‘MyModelstat’) = MyModel.modelstat;
RunData(xx,‘MySolvestat’) = MyModel.solvestat;
RunData(xx,‘MyTime’) = MyModel.resusd;
RunData(xx,‘Pgen1’) = PGen.l(‘g00001’);
{bunch of other code capturing whatever vars I’m interested
in}
yy = yy + 1;
if (round(yy/100) = yy/100,
execute_unload ‘ParamTestOutput’;
);
if (ACOPF.modelstat = 2,
execute_unload ‘Success’;
);
);

It should be pretty obvious what this does. At the end, I can open
ParamTestOutput and scroll through the RunData table (or copy it to
Excel to do some analysis about different solutions, run times, etc.).

I do a LOT of these, so I want to use all 8 of my processors to speed
things up, if I can. I think I understand that I can do that using
solvestat = 3. As I said, I reviewed the McCarl guide, but can’t
quite figure out what’s going on there. Any help would be great.

I now have the " if (round(yy/100) = yy/100…" code in there so that
it stores the results every 100 runs (so that I can see what’s going
on without waiting for thousands or hundreds of thousands of runs to
complete). That’s not necessary, but if I could have some feature
like that in my grid-enabled code, it would really help. Or do I just
n

By the way, I’ve asked a number of other researchers who use GAMS
pretty regularly, and they were all very interested in this issue, but
weren’t able to understand the user guide here either. So you’ll be
helping several people, as they wanted me to forward the answer to
them as soon as I have it.

Thanks much!
Emily

–
“gamsworld” group.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group athttp://groups.google.com/group/gamsworld?hl=en.

model.gms
1KViewDownload

driver.gms
1KViewDownload

doload.gms
< 1KViewDownload

\