GDX GAMS MatLab rgdx dimensions obtaining unnecessary large and sparse matrices

GAMS
1

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing. Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse data that contains lots of 0s. To be more specific I noticed that data is just filled up to the number of elements corresponding to the largest set. That is to say, if I have three sets i,j,k with cardinality Ni<Nj<Nk then from the GDX files all data now has cardinality Nk. For instance if I read a variable that is only defined in dimensions i,j, then I obtain a k-x-k-matrix , where there is only sensible data for the i-x-j-submatrix. Accordingly it is very cumbersome to extract the data in the form of the submatrices. I wonder if there is an automated way of telling the rgdx only to read the sensible data?

Many thanks for any advice.

-imo

–
To view this discussion on the web visit https://groups.google.com/d/msg/gamsworld/-/GVF4vIHyZT0J.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.

2

There is an option for that!

I attach below a data file, and a readSparse.m file that we use for testing (its a bit verbose but shows what we expect to get back, and then checks we do in fact get it back from the rgdx call). It shows how to use the compress feature and lots of other options in the reading of gdx files into matlab via rgdx.

The key lines for you are (I believe):

clear rFlags;
rFlags.name = ‘I’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

then has the 14 or so uels that are in the “global uel list” from the readFull.gdx file.

If you add the “compress” flag, then the interface squashes out those uels.

rFlags.compress = ‘true’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

gives what I think you wanted. You may want to experiment with some other more complex sets or parameters to ensure you get back what you are expecting.

Cheers, Michael





On Jun 6, 2012, at 11:34 AM, imo wrote:

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing. Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse data that contains lots of 0s. To be more specific I noticed that data is just filled up to the number of elements corresponding to the largest set. That is to say, if I have three sets i,j,k with cardinality Ni >
Many thanks for any advice.

-imo

–
readFull.gdx (17.3 KB)
readSparse.m (18.3 KB)

3

Many thanks for your advice and the files you provide.

Unluckily, the compress flag does not solve the problem. This option in the rgdx routine allows a user to remove ALL rows and ALL columns with all zeros from the ‘val’ data matrix but SOME of those rows are desired. Let me explain with the example from above: reading a variable that is only defined in dimensions i,j, and obtaining a k-x-k-matrix, while what is desired is only the sensible data for the i-x-j-submatrix, the compress tag would delete rows and columns in the submatrix that are zero even though they may be sensible data from the optimal solution of that variable…

Cheers,
+i

On Wednesday, June 6, 2012 6:30:38 PM UTC+1, Michael Ferris wrote:

There is an option for that!

I attach below a data file, and a readSparse.m file that we use for testing (its a bit verbose but shows what we expect to get back, and then checks we do in fact get it back from the rgdx call). It shows how to use the compress feature and lots of other options in the reading of gdx files into matlab via rgdx.

The key lines for you are (I believe):

clear rFlags;
rFlags.name = ‘I’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

then has the 14 or so uels that are in the “global uel list” from the readFull.gdx file.

If you add the “compress” flag, then the interface squashes out those uels.

rFlags.compress = ‘true’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

gives what I think you wanted. You may want to experiment with some other more complex sets or parameters to ensure you get back what you are expecting.

Cheers, Michael




On Jun 6, 2012, at 11:34 AM, imo wrote:

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing. Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse data that contains lots of 0s. To be more specific I noticed that data is just filled up to the number of elements corresponding to the largest set. That is to say, if I have three sets i,j,k with cardinality Ni
Many thanks for any advice.

-imo

–
To view this discussion on the web visit https://groups.google.com/d/msg/gamsworld/-/GVF4vIHyZT0J.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.

–
To view this discussion on the web visit https://groups.google.com/d/msg/gamsworld/-/1AH_gxPMC0AJ.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.

4

Ilan,

The attached files illustrate a simple way to get the data back in a
matrix with the size you expect, i.e. as an i-x-j matrix, not a -x-
matrix. This code works with recent GAMS Distributions. The .gdx and
.m files are all you need: the .gms file was used to generate the .gdx
and is just there FYI.

One of the issues that comes into play here is that the domain
information is not always stored in the GDX. You may write a
parameter A(I,J) to GDX along with I and J but the connection between
these three was not always saved. This is changing: we are doing more
to store the domain information in the GDX with our next release 23.9
than we’ve done previous to that. With this information it becomes
possible to return an i-x-j matrix by default. We plan to update
GDXMRW to do this in the near future.

-Steve

On Tue, Jul 3, 2012 at 6:04 AM, imo wrote:

Many thanks for your advice and the files you provide.

Unluckily, the compress flag does not solve the problem. This option in the
rgdx routine allows a user to remove ALL rows and ALL columns with all zeros
from the ‘val’ data matrix but SOME of those rows are desired. Let me
explain with the example from above: reading a variable that is only defined
in dimensions i,j, and obtaining a k-x-k-matrix, while what is desired is
only the sensible data for the i-x-j-submatrix, the compress tag would
delete rows and columns in the submatrix that are zero even though they may
be sensible data from the optimal solution of that variable…

Cheers,
+i

On Wednesday, June 6, 2012 6:30:38 PM UTC+1, Michael Ferris wrote:

There is an option for that!

I attach below a data file, and a readSparse.m file that we use for
testing (its a bit verbose but shows what we expect to get back, and then
checks we do in fact get it back from the rgdx call). It shows how to use
the compress feature and lots of other options in the reading of gdx files
into matlab via rgdx.

The key lines for you are (I believe):

clear rFlags;
rFlags.name = ‘I’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

then has the 14 or so uels that are in the “global uel list” from the
readFull.gdx file.

If you add the “compress” flag, then the interface squashes out those
uels.

rFlags.compress = ‘true’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

gives what I think you wanted. You may want to experiment with some
other more complex sets or parameters to ensure you get back what you are
expecting.

Cheers, Michael

On Jun 6, 2012, at 11:34 AM, imo wrote:

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing.
Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse
data that contains lots of 0s. To be more specific I noticed that data is
just filled up to the number of elements corresponding to the largest set.
That is to say, if I have three sets i,j,k with cardinality Ni>> >> > from the GDX files all data now has cardinality Nk. For instance if I read a
variable that is only defined in dimensions i,j, then I obtain a
k-x-k-matrix , where there is only sensible data for the i-x-j-submatrix.
Accordingly it is very cumbersome to extract the data in the form of the
submatrices. I wonder if there is an automated way of telling the rgdx only
to read the sensible data?

Many thanks for any advice.

-imo

–
You received this message because you are subscribed to the Google
Groups “gamsworld” group.
To view this discussion on the web visit
https://groups.google.com/d/msg/gamsworld/-/GVF4vIHyZT0J.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.

–
readSol.zip (1.84 KB)

5

Many thanks Steve, your code was indeed VERY helpful.

The code has been working just fine although I still find it a bit crude creating so many structs (2 per set and then again 2 per output variable). However, since just before X-mas there is the new GAMS version 24.0, I was wondering if there is an even better way to do it now. Will you let me know if there is anything that makes life even easier?

Cheers,
+i

On Tuesday, July 3, 2012 10:00:53 PM UTC+2, Steven Dirkse wrote:

Ilan,

The attached files illustrate a simple way to get the data back in a
matrix with the size you expect, i.e. as an i-x-j matrix, not a -x-
matrix. This code works with recent GAMS Distributions. The .gdx and
.m files are all you need: the .gms file was used to generate the .gdx
and is just there FYI.

One of the issues that comes into play here is that the domain
information is not always stored in the GDX. You may write a
parameter A(I,J) to GDX along with I and J but the connection between
these three was not always saved. This is changing: we are doing more
to store the domain information in the GDX with our next release 23.9
than we’ve done previous to that. With this information it becomes
possible to return an i-x-j matrix by default. We plan to update
GDXMRW to do this in the near future.

-Steve

On Tue, Jul 3, 2012 at 6:04 AM, imo wrote:

Many thanks for your advice and the files you provide.

Unluckily, the compress flag does not solve the problem. This option in the
rgdx routine allows a user to remove ALL rows and ALL columns with all zeros
from the ‘val’ data matrix but SOME of those rows are desired. Let me
explain with the example from above: reading a variable that is only defined
in dimensions i,j, and obtaining a k-x-k-matrix, while what is desired is
only the sensible data for the i-x-j-submatrix, the compress tag would
delete rows and columns in the submatrix that are zero even though they may
be sensible data from the optimal solution of that variable…

Cheers,
+i

On Wednesday, June 6, 2012 6:30:38 PM UTC+1, Michael Ferris wrote:

There is an option for that!

I attach below a data file, and a readSparse.m file that we use for
testing (its a bit verbose but shows what we expect to get back, and then
checks we do in fact get it back from the rgdx call). It shows how to use
the compress feature and lots of other options in the reading of gdx files
into matlab via rgdx.

The key lines for you are (I believe):

clear rFlags;
rFlags.name = ‘I’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

then has the 14 or so uels that are in the “global uel list” from the
readFull.gdx file.

If you add the “compress” flag, then the interface squashes out those
uels.

rFlags.compress = ‘true’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

gives what I think you wanted. You may want to experiment with some
other more complex sets or parameters to ensure you get back what you are
expecting.

Cheers, Michael

On Jun 6, 2012, at 11:34 AM, imo wrote:

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing.
Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse
data that contains lots of 0s. To be more specific I noticed that data is
just filled up to the number of elements corresponding to the largest set.
That is to say, if I have three sets i,j,k with cardinality Ni> > from the GDX files all data now has cardinality Nk. For instance if I read a
variable that is only defined in dimensions i,j, then I obtain a
k-x-k-matrix , where there is only sensible data for the i-x-j-submatrix.
Accordingly it is very cumbersome to extract the data in the form of the
submatrices. I wonder if there is an automated way of telling the rgdx only
to read the sensible data?

Many thanks for any advice.

-imo

–
You received this message because you are subscribed to the Google
Groups “gamsworld” group.
To view this discussion on the web visit
https://groups.google.com/d/msg/gamsworld/-/GVF4vIHyZT0J.
To post to this group, send email to gams...@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.

–
“gamsworld” group.
To view this discussion on the web visit
https://groups.google.com/d/msg/gamsworld/-/1AH_gxPMC0AJ.

To post to this group, send email to gams...@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.



–
Steven Dirkse, Ph.D.
GAMS Development Corp., Washington DC
Voice: (202)342-0180 Fax: (202)342-0181
sdi...@gams.com
http://www.gams.com

–
To view this discussion on the web visit https://groups.google.com/d/msg/gamsworld/-/7MtL528vLcsJ.
To post to this group, send email to gamsworld@googlegroups.com.
To unsubscribe from this group, send email to gamsworld+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/gamsworld?hl=en.

6

Thanks Steven, actually I had the same problem as imo and used your readSol.m file to solve my problem.

On Wednesday, July 4, 2012 12:00:53 AM UTC+4, Steven Dirkse wrote:

Ilan,

The attached files illustrate a simple way to get the data back in a
matrix with the size you expect, i.e. as an i-x-j matrix, not a -x-
matrix. This code works with recent GAMS Distributions. The .gdx and
.m files are all you need: the .gms file was used to generate the .gdx
and is just there FYI.

One of the issues that comes into play here is that the domain
information is not always stored in the GDX. You may write a
parameter A(I,J) to GDX along with I and J but the connection between
these three was not always saved. This is changing: we are doing more
to store the domain information in the GDX with our next release 23.9
than we’ve done previous to that. With this information it becomes
possible to return an i-x-j matrix by default. We plan to update
GDXMRW to do this in the near future.

-Steve

On Tue, Jul 3, 2012 at 6:04 AM, imo wrote:

Many thanks for your advice and the files you provide.

Unluckily, the compress flag does not solve the problem. This option in the
rgdx routine allows a user to remove ALL rows and ALL columns with all zeros
from the ‘val’ data matrix but SOME of those rows are desired. Let me
explain with the example from above: reading a variable that is only defined
in dimensions i,j, and obtaining a k-x-k-matrix, while what is desired is
only the sensible data for the i-x-j-submatrix, the compress tag would
delete rows and columns in the submatrix that are zero even though they may
be sensible data from the optimal solution of that variable…

Cheers,
+i

On Wednesday, June 6, 2012 6:30:38 PM UTC+1, Michael Ferris wrote:

There is an option for that!

I attach below a data file, and a readSparse.m file that we use for
testing (its a bit verbose but shows what we expect to get back, and then
checks we do in fact get it back from the rgdx call). It shows how to use
the compress feature and lots of other options in the reading of gdx files
into matlab via rgdx.

The key lines for you are (I believe):

clear rFlags;
rFlags.name = ‘I’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

then has the 14 or so uels that are in the “global uel list” from the
readFull.gdx file.

If you add the “compress” flag, then the interface squashes out those
uels.

rFlags.compress = ‘true’;
r = rgdx (‘readFull.gdx’, rFlags);
r.uels{1}

gives what I think you wanted. You may want to experiment with some
other more complex sets or parameters to ensure you get back what you are
expecting.

Cheers, Michael

On Jun 6, 2012, at 11:34 AM, imo wrote:

Hi,

I am working with a GAMS MatLab hybrid via gdx for interfacing.
Specifically with: rgdx() and wdgx().

When reading the GAMS output in Matlab with rgdx() I obtain very sparse
data that contains lots of 0s. To be more specific I noticed that data is
just filled up to the number of elements corresponding to the largest set.
That is to say, if I have three sets i,j,k with cardinality Ni> > from the GDX files all data now has cardinality Nk. For instance if I read a
variable that is only defined in dimensions i,j, then I obtain a
k-x-k-matrix , where there is only sensible data for the i-x-j-submatrix.
Accordingly it is very cumbersome to extract the data in the form of the
submatrices. I wonder if there is an automated way of telling the rgdx only
to read the sensible data?

Many thanks for any advice.

-imo

–
You received this message because you are subscribed to the Google
Groups “gamsworld” group.
To view this discussion on the web visit
https://groups.google.com/d/msg/gamsworld/-/GVF4vIHyZT0J.
To post to this group, send email to gams...@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.

–
“gamsworld” group.
To view this discussion on the web visit
https://groups.google.com/d/msg/gamsworld/-/1AH_gxPMC0AJ.

To post to this group, send email to gams...@googlegroups.com.
To unsubscribe from this group, send email to
gamsworld+...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/gamsworld?hl=en.



–
Steven Dirkse, Ph.D.
GAMS Development Corp., Washington DC
Voice: (202)342-0180 Fax: (202)342-0181
sdi...@gams.com
http://www.gams.com

–
To unsubscribe from this group and stop receiving emails from it, send an email to gamsworld+unsubscribe@googlegroups.com.
To post to this group, send email to gamsworld@googlegroups.com.
Visit this group at http://groups.google.com/group/gamsworld.
For more options, visit https://groups.google.com/groups/opt_out.