Manipulatiing parameters - Assignment problem due to different dimension

GAMS
1

Hi,
I’m having trouble assigning values. For example I have several files and every file represents a sation which meassures windspeed. These files look like this:

Station;Date;Time;  SP
       34;2017-09-12;0000;    2.5
       34;2017-09-12;0010;    6.1
       34;2017-09-12;0020;    3.0
       ...

Then I have other files where power is assigned to speed depending on the generator type

type;Speed;Power
1;       0	;0
1;       0.1	;0
...
 1;      2.9	;0
 1;      3	;100
 1;      3.1	;211
 ...
 1;      34.9	;4000
 1;      35	;0
 1;      35.1	;0
...

Now I want to calculate the power output for every station at each day and time and for a specific typ, so the gdx file should look something like this:

i;j;k;t; value
       34;2017-09-12;0000; 1;   0
       34;2017-09-12;0010; 1;   1050
       34;2017-09-12;0020; 1;   100
       ...
       34;2017-09-12;0000; 2;   0
       34;2017-09-12;0010; 2;   2111
       34;2017-09-12;0020; 2;   300

My code is attached, but it won’t work (error - dimension different).

Thanks in advanced for your help.
Jason

PS: If someone happen to know if there is an easy way to read several input files(or could point me in the right direction) I would appreciate it too. I’ve read about the Put Writing Facility, but at first impression it was not very intuitive. I was thinking more of a simple loop, but that doesn’t seem to work or at least I dont know how.
power.gms (1.11 KB)

2

Hi,

If I understand you correctly, you somehow need to compare the set elements of

set    w 'windspeed';

to the parameter values of

Parameter speed(i,j,k)         'Speed';

In your approach you basically try to replace set w with parameter speed(i,j,k) which - as you already noticed - does not work that way. However, if you know that set elements represent numerical values (such as the the elements of w), you can access those values via set attribute .val (https://www.gams.com/latest/docs/UG_SetDefinition.html#UG_SetDefinition_SetAttributes). The following should do the trick:

power_output(i,j,k,t) = sum(w$[abs(w.val-speed(i,j,k))<1e-6], power_wind(t,w));

In the example I check whether the absolute difference between w.val and parameter speed is < 1e-6.

Regarding your question on how to read multiple input files:
It seems that if you have stations 1,2,…,N you want to read files station1.txt, station2.txt, …, stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.

One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what’s the reason of having the data in N files instead of having it in just one file.

Best,
Fred

3

Hi Fred,

thank you very much for your reply. Your code example did what I wanted, thanks! I never would have guessed. There is not an easier solution to such a problem? I don’t mean that in a bad way, I’m just curious, because I thought you can solve it with a simple assignment also with the ulterior motive that this would be faster too (Speed is not an issue yet, but it took GAMS 30s to execute this step - 160.000 entries were created in this step, in comparison reading 4 files with 80.000 entries each took like 1s)

It seems that if you have stations 1,2,…,N you want to read files station1.txt, station2.txt, …, stationN.txt. put_utulity, as you already suspected, allows you to do that. If you share a small executable example that illustrates what you want to do, other users might use that example and modify it. That would make helping you much easier than constructing examples from a few code snippets.

I’m not sure if I understand you correctly, but I wasn’t able to get into the put utulity and came up with a solution or at least something close to a solution, yet. Or do you mean such examples?
Generator1.txt (10.5 KB)
Station1.txt (1.77 MB)

One more remark: it Seems that for example in station*.txt, the actual station is not only part of the filename but also defined via the first column of the file. Given that, I wonder what’s the reason of having the data in N files instead of having it in just one file.

I get the data from an external source, so I don’t have any influence on it. Maybe it would be simply too much data for a regular user who just needs data from one station, but I dont know.

Greetings,
Jason

4

Jason,

Sorry for the late reply.
Seems that we are discussing two points

1. Slow execution of assignment
Let me try to give you some background. You use different data types for wind speed.
a) set w; This set contains different wind speeds in the form of a discrete set 0.0, 0.1, 0.2, 0.3, … Set elements are treated as strings (but if you know they have a numerical value, you can access it via .val)
b) parameter speed(i,j,k); This parameter also contains different wind speeds but as numerical values 0.0, 0.1, 0.2, 0.3, …

Since you already treat wind speeds as set elements, I wonder whether you actually need parameter speed. You could read the wind speeds as sets from the station*.txt files. The advantage is that this allows you to get rid of the comparison (abs(w.val-speed(i,j,k))<1e-6) in the assignment to power_output(i,j,k,t) and do sth. like

...
set speed(i,j,k,w)
...
power_output(i,j,k,t) = sum(speed(i,j,k,w), power_wind(t,w));
...

This is considerably faster.


2. reading enumerated files with identical filestem (station1.txt, station2.txt, …)
There are many ways to reduce the amount of code. The attached examples illustrate two alternatives. Note that your recent station.txt is slightly different to the on from post #1. I read only some of the columns of station*.txt and do not read generator*.txt but I guess you can adopt the examples as needed.

a) batinclude: Write a little GAMS script that takes the number of the file as argument and does the reading.

...
$onechoV > readStationData.gms
$call csv2gdx station%1.txt output=station%1.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y
$ife errorlevel<>0 $abort problem with csv2gdx station%1
$gdxIn station%1.gdx
$load speed
$gdxIn
$offecho

$onMulti
$batinclude readStationData.gms 1
$batinclude readStationData.gms 2
...

$onecho … writes the file readStationData.gms which us subesequently (bat)included. The argument of the batinclude replaces the internal placeholder (%1). You can read more about that in the documentations


b) put_ultility: Write a script where in the script you loop over all the files that need to be read.

...
$onechoV > readStationData.gms
file load / loadGDX.inc /;
set f files / 1*%NBFILES%/;
loop(f,
  put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'
  abort$(errorlevel<>0) 'probem running csv2gdx via put utility'
  put load '$gdxin station' f.tl:0 '.gdx' /
           '$load speed'                  /
           '$gdxin'                       /;
);
putclose;

$offecho

$call gams readStationData.gms lo=%gams.lo% --NBFILES=5
$ife errorlevel<>0 $abort problem running readStationData.gms
$onMulti
$include loadGDX.inc
...

You pass the number of files to read (–NBFILES) as an argument to the script which loops over the txt files, creates the gdx files, and writes a list of gdxin instructions to include later. Note that you have to outsource this put_utility approach to a script because the put_utility command is carried out at execution time while you (probably) want the data to be available at compile time. The trick is to call the script at compile time via $call. You can read more about the two phases (compilation, execution) in the documentation: https://www.gams.com/latest/docs/UG_GamsCall.html#UG_GamsCall_TwoPass


I hope this helps!

Fred
jason.zip (772 KB)

5

Fred, you are awesome. Thank you so much for your detailed answer!!

I used the put ultility and I got it working :slight_smile:

Not that important, but did you use this

put_utility 'exec' / 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y'

instead of

put load 'csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /

on purpose?
Because both works, but I wonder if there is a significant difference between them or is the latter maybe just a bad coding style?

6

Thanks :slight_smile:

In fact there is significant difference and I used put_utility on purpose.

With put_utility the csv2gdx command is executed from readStationData.gms which you call at compile time of your main file → Data is available at compile time in your main file and you can do $gdxin, $load etc.

With put the csv2gdx command is written to file loadGDX.inc which is included in your main file. In general, you could do that. Then you need to run the csv2gdx command at compile time via $call. In your example, it seems that you don’t run it at all, so you should get an error. The following should work:

[...]
$onechoV > readStationData.gms
file load / loadGDX.inc /;
set f files / 1*%NBFILES%/;
loop(f,
  put load '$call csv2gdx station' f.tl:0 '.txt output=station' f.tl:0 '.gdx id=speed fieldSep=semiColon index=1,2,3,5 useHeader=y storeZero=y' /
           '$ife errorlevel<>0 $abort problem with csv2gdx station'f.tl:0 /
           '$gdxin station' f.tl:0 '.gdx' /
           '$load speed'                  /
           '$gdxin'                       /;
);
putclose;
$offecho

$call gams readStationData.gms lo=%gams.lo% --NBFILES=5
$ife errorlevel<>0 $abort problem running readStationData.gms
$onMulti
$include loadGDX.inc
[...]

Best,
Fred

7

Alright thanks again for the clarification. I get it now. I had the put_utility command and the put command not differentiated, for me both fell under the category put_utility.
Best regards,
Jason