Hi Arne Drud,
Thank you for your assessment and comments. I now know that I have to invest in the model so I can keep the million dollars for me instead of giving it to the memory.
All the questions you are making are relevant and the reply is: Yes to all. And I understand the point you are making. Entities are major petroleum companies in a petroleum supply chain and the problem is a petroleum supply chain design problem.
I will take your advice and work small scale again to fine trim the model before getting back to the large scale data.
Thanks again.
Melhores Cumprimentos,
LeÃ£o Fernandes.
From: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] On Behalf Of Arne Stolbjerg Drud
Sent: quintafeira, 5 de Maio de 2011 13:01
To: gamsworld@googlegroups.com
Subject: SV: Model takes a lot of time to generate
Hi LeÃ£o Fernandes
Try to use consider your own numbers. As you describe it you will have 18751201 = 75600 mass balance equations. On the receiving side you have 410 = 40 variables and on the delivery side 418 = 72 variables in each mass balance equation. Since the variables are all different you will have a total of 3,024,000 variables on the receiving side and 5,443,200 variables on the delivery side or a total of 8,467,200 variables. Before adding scenarios that will give another multiplicative factor. And I assume that this is just part of your model. This is not large scale â€“ this is EXTREME scale.
As others have commented, you will have to constrain your model by adding conditions. Typically you should look at questions like these:

Does all depots handle all products?

Is there a relationship between products and entities (I do not know what you mean)?

Will all plants deliver to all depots?

Will all depots deliver to all clients?

Is there a relationship between transport mode and products?

Etc etc.
You ask if you should not be using so much memory. My answer is that if you can get rid of unnecessary variables and equations then you will not use so much memory. But if you insist on have all combinations of everything in your model, then you (and GAMS) will need a million dollarsâ€™ worth of memory to generate the model. If it can be solved afterwards is then then another question.
Spend the days it takes to get your smallscale model in shape before trying a large case again.
Regards
Arne Stolbjerg Drud
\
Arne Stolbjerg Drud
ARKI Consulting & Development A/S
Bagsvaerdvej 246A, DK2880 Bagsvaerd, Denmark
Phone: (+45) 44 49 03 23, Fax: (+45) 44 49 03 33, email: adrud@arki.dk
Fra: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] PÃ¥ vegne af LeÃ£o Fernandes
Sendt: 5. maj 2011 13:09
Til: gamsworld@googlegroups.com
Emne: RE: Model takes a lot of time to generate
Hello Arne Drud,
Thank you for your reply.
The line number is an excellent help which I did not know or overlooked. The constraint on this line will perhaps generate the most combinatory situations. Unfortunately it is not the only one as I see that the generation of each line takes long.
This constraint in particular is a mass balance equation which says that:
the initial stock at depot + product receival at depot = product delivery to other depots + product delivery to clients + final stock at depot.
The depot mass balance equation is enforced for all possible depots (18), products (7), entities (5), periods (120) and scenarios (1).
The receival quantity variables have two more dimensions transport mode (4) and plants (10) in addition to the above.
The delivery variables also have two additional dimensions transport mode (4) and clients (18) which is something like sum((k,m), QS(j,k,m,p,e,t,s))
So far I am only using 1 scenario, so the model will explode further, so I have to get it under control.
This means perhaps that I have to look at this statement and develop some cuts.
In 32 bit version of GAMS I used to get the out of memory error but had the same symptoms mentioned earlier.
In 64 bit version, after 24 hours, it has not completed generation of the model, so I interrupted it.
I agree that the memory is full and the machine goes slow so I cannot use it to write into a word document, and I see it is busy not writing but reading from disk, which I donot know what.
Can I assume from your reply, that I should not have been using so much memory and that it is normal for GAMS to start writing to disk from memory and then consequently the processor will drop down to 1% usage and 99% idle as the computer is more busy writing to disk than processing. That too during one full day.
I know that I must have a problem with my model whose memory usage could be reduced. However this performance of GAMS does not look to be normal even for a largescale problem, which frankly I do not know if it is.
However I would not expect the generation of the model to take so long. I would expect the execution to take long.
I would like to receive comments as to where and what I could start looking at. Sorry that I may be asking too much, but I am still quite new to GAMS and perhaps it is easier for an expert to make more sense from the information I am sending here.
Melhores Cumprimentos,
LeÃ£o Fernandes.
From: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] On Behalf Of Arne Stolbjerg Drud
Sent: quartafeira, 4 de Maio de 2011 15:59
To: gamsworld@googlegroups.com
Subject: SV: Model takes a lot of time to generate
Hi LeÃ£o Fernandes
If you look at the line in the bottom left corner of the GAMS screen it tells you which line is being generated (1318) and how much memory you are using (9828 Mbytes). The amount of memory is larger than your physical machine so it starts to swap, and memory on disk is MUCH slower than Ram.
You must reduce the amount of memory you use. The McCarlguide in the Help memnu has some comments of this â€“ just search for â€œmemoryâ€.
Regards
Arne
\
Arne Stolbjerg Drud
ARKI Consulting & Development A/S
Bagsvaerdvej 246A, DK2880 Bagsvaerd, Denmark
Phone: (+45) 44 49 03 23, Fax: (+45) 44 49 03 33, email: adrud@arki.dk
Fra: gamsworld@googlegroups.com [mailto:gamsworld@googlegroups.com] PÃ¥ vegne af LeÃ£o Fernandes
Sendt: 4. maj 2011 16:02
Til: gamsworld@googlegroups.com
Emne: Model takes a lot of time to generate
Hello Everybody,
I have an interesting problem which I have been battling against for some time.
I am running a MIP model on a multiprocessor PC with 64 bit windows 7 and Gams 22.3 and ILOG CPLEX 12.1 MIP solver.
I have a problem that has happened many times when I go from a small example to a largescale example.
So far I have survived by reducing the data set by reducing the number of periods so I can complete the run of the model, but when I increase the time periods from about 12 months (which takes arround 10 minutes) to arround 120 that is a realcase setting this is what happens.
I find that the processor gets about 100% busy, then I find the memory gradually increasing and gets over 90% ocupied or so. Simultaneously the processor usage drops to arround 2% of processor capacity and then it will takes hours and days to complete the job.
All this happens when GAMS is still generating the model. Maybe the processing slows down because of memory ocupation, and as a consequence takes an ieternity to complete.
I have 8Gbytes of memory and 8 processores and still have problems. I need to get over this problem so I can fully test my model.
Can anybody share some light of what might be happening and how I may resolve the situation. This is the print of the GAMS screen.
Thank You,
LeÃ£o Fernandes.
–
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.
–
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.
–
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.
–
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.
–
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.