When I am using embedded python code in GAMS can I import python packages such as pandas some how? When I am just normally write :
$onEmbeddedCode Python:
import pandas as pd
I get an 865 Problem in Embedded Code Section
Thank you
Yes you can. Either install the package manually by navigating to your GMSPython folder (i.e. /GMSPython) and using the pip executable (python package management system). This tool should be located in a folder “Scripts” or something similar. (you can install pandas with pip with the command: “pip install pandas”) Alternatively use the solution proposed here to import packages from within your code: https://stackoverflow.com/questions/12332975/installing-python-module-within-code
Hi
I had the facility for running embedded code with python up and running, but now I am on my new computer and my short term memory was also cleared: I haven’t a clue how I got it working.
Embedded code runs fine, but if I add “import pandas as pd”, I get the error that pandas is not installed.
— Initialize embedded library embpycclib64.dll
— Execute embedded library embpycclib64.dll
— Exception from Python: No module named ‘pandas’
*** Error executing Python embedded code section:
If I go to the GMSPython directory and issue:
pip install pandas
it uses my Anaconda python installation and tells me pandas is already installed.
I use Anaconda 3.6 on Windows 10. The path points to the anaconda installation.
I ran the command “python setup.py install” in the 3.6 api directory of Gams.
Any help would be appreciated.
Cheers
Renger
Renger, use the pip executable that comes with GMSPython. It is located in the Scripts subfolder:
c:\gams\win64\26.1\GMSPython\Scripts\pip install pandas
This should do it.
-Michael
That worked. However, is it possible to use anaconda instead of Python coming with GAMS? I ask because I can’t use matplotlib on Windows because of the missing module tkinter. It looks like Gams python is a scaled down version.
Cheers
Renger
Renger,
You are right. Tkinter is not part of GMSPython. If you only want to get matplotlib running, there is a workaround for this. You can use wxPython. Install it using: pip install wxPython and then in embeddedCode you can do the following:
import matplotlib
matplotlib.use('WXAgg')
import matplotlib.pyplot as plt
...
Using a different Python interpreter is also possible but has some limitations. The most important one is that it only works with Python 3.6. You can find the details here: https://www.gams.com/latest/docs/UG_EmbeddedCode.html#UG_EmbeddedCode_PortingToDifferentVersionOfPython
Hope that helps,
Clemens
Hi Clemens
Thanks, but I don’t manage to get it running (the wxAgg doesn’t do the job either as I want to have a plot showing up).
Could you help me out (all on Windows 10 x64):
- I ran the setup.py script in the 3.6 directory
- I have my Python 3.6 in c:\users\renger\anaconda (there is also the 3.6.dll) and added this to my path.
- I have no new user variables defined (I tried a few like PYTHONPATH set to either the anaconda directory or to C:\GAMS\apifiles\Python\api_36).
I tried running gams python.gms --pySetup=0
--- Execute embedded library embpycclib64.dll
--- Exception from Python: No module named 'tkinter'
--- python.gms(1) 3 Mb 1 Error
*** Status: Execution error(s)
where python.gms is:
embeddedCode Python:
import matplotlib
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
plt.ylabel('some numbers')
plt.show()
endEmbeddedCode
Could you point me to the correct settings?
Cheers
Renger
Renger,
Regrading wxPython: I have attached the file
meanvarPlot.gms
which shows how to use Embedded Code and matplotlib/wxPython for generating a plot which shows up in a new window. Your Python interpreter needs to have both matplotlib and also wxPython installed.
Regarding PySetup=0: There seems to be a problem which might prevent this feature to work properly. The GAMS system directory contains a Python36.dll as well which is found by Embedded Code even though you specify PySetup=0. The only workaround that comes to my mind is to rename the Python36.dll in order to prevent it from being found. Note that this might introduce problems with certain tools (e.g. model2tex) which require the DLL in this location. Then modify your PATH environment variable before calling GAMS in the following way:
set PATH=c:\users\renger\anaconda\;%PATH%
Please let me know if one of the suggestions work for you
Best,
Clemens
Hi Clemens
After installing wxPython, I can call the plot, so that is fine.
However, even after renaming the python3.6.dll in the Gams root directory, I still can’t use my ananconda installation. For whatever reason, gams seems to still have problems with a missing tkinter.
Anyway, I can now work with the plots. Thanks!
Perhaps that the Python-Nerds are used to having virtual environments for every project and don’t care about using several versions of installed versions (Gams, 3.6, 2.5, etc.). I am used to R and there you usually rely on just one installation. But it might be necessary for Python, as some packages don’t run on the newest releases (which happens hardly using R…)
Cheers
Renger
Good to hear that the plotting is working.
Perhaps one could start from a setup where no Python is found at all. This means that the python36.dll in the GAMS system directory is renamed. Furthermore the PYTHONPATH and also the PATH should be empty or at least should not contain any PYTHON or GAMS specific information. A GAMS run with pysetup=0 should then fail in any case. As a second step, the location of the python36.dll in the Anaconda version of Python is added to the PATH which should result in embedded code to work. As a third step we can try to import Tkinter (which should work as well).
Best,
Clemens
Hi Clemens
The story continues: I now have Gams on my Ubuntu 18 notebook installed and run in the same kind of problems. I use Anaconda and the following Gams-code:
embeddedCode Python:
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('WXagg')
plt.plot([1,2,3,4])
plt.show()
endEmbeddedCode
This gives me following error:
--- Initialize embedded library libembpycclib64.so
--- Execute embedded library libembpycclib64.so
--- Exception from Python: No module named 'wx'
*** Error at line 1: Error executing "embeddedCode" section: Check log above
--- python.gms(1) 3 Mb 1 Error
*** Status: Execution error(s)
--- Job python.gms Stop 03/04/19 17:20:58 elapsed 0:00:00.857
When I check the version and which python, I have this:
(base) renger@ubu:~$ which python
/home/renger/anaconda3/bin/python
I installed wxPython for Ananaconda using: conda install -c anaconda wxpython
I ran python3 setup.py install (and all was installed in the anaconda directory: home/renger/anaconda3/lib/python3.6/site-packages/
Any idea how to resolve this?
Cheers
Renger
PS. I am already looking out for the experience of installing this on the ETH mainframe…
Renger,
The mechanism on finding the Python interpreter is different on Linux. The shared object that is responsible for loading the Python interpreter uses @rpath to resolve its dependency to the Python shared object. Therefore it will be very hard to convince Embedded Code to use a different Python interpreter as long as the GMSPython directory is present. The solution for this is similar to the workaround on Windows. We need to make sure that the GMSPython directory is not found (e.g. by renaming). In addition we need to adjust the LD_LIBRARY_PATH (DYLD_LIBRARY_PATH on Mac OSX) to point to the location of libpython3.6m.so.1.0. In order to find required modules, setting the PYTHONPATH is perhaps required as well.
Let me know if you can get it running,
Clemens
Hi Clemens
I gave up as I wasn’t able to get it running (Python in GAMS yes, but plotting no).
Even if I set the python path to GMSPython I run into lots of problems and did not manage to install wxpython.
My wishes:
-
- It would be great if the information on how to do this would be clear even for Linux-Dummies like me. Either using the GMSPython or the own-installed Python version. I prefer my “own” Python instead of having multiply Python versions lying around.
- Or prepare the GAMS python setup in such a way that one could run (on-the-fly) plots in the embedded code (at least have some packages preinstalled for plotting).
Cheers
Renger
Hi Clemens
Here are my steps on my Windows machine to get the embedded code working with:
- Install the python distribution of your liking (as long as it is version 3.6.x; not one of the versions 3.7 or 3.5)
- Adjust your path by adding the path to the python3.dll (in my case: C:\Python36\python-3.6.8.amd64) and to the scripts directory of your distribution (e.g. C:\Python36\python-3.6.8.amd64\Scripts). This can be done under “Control panel” - “System” → “Advanced system settings” → “Environmental Variables” → scroll to “path” and add these two new entries. If you don not have administrator rights you can add it to the user path in the upper window.
- Rename the file python3.dll in the Gams root directory, so Gams can’t use it.
- Rename in gmspython subdirectory of Gams the files python.exe, pythonw.exe, python3.dll, and python36.dll.
- Open a dos box (cmd) and run in the directory of the 3.6 apifile (C:\GAMS..\apifiles\Python\api_36) the following command “python setup.py install”.
- With the following lines in embedded code you can check if Gams is using your distribution and not the Gams version (3.6.1). It saves a file with the version number in the directory of your Gams program with embedded code.
embeddedCode Python:
import sys
file = open('version.txt', 'w')
version1 = str(sys.version_info[0])
version2 = str(sys.version_info[1])
version3 = str(sys.version_info[2])
file.write(version1+'.'+version2 + '.' + version3)
file.close()
endEmbeddedCode
- If it doesn’t work, don’t ask me, ask Clemens
PS. Michael Bussieck informed me that in the new version of Gams, for Windows you only have to use the option pysetup=0. If you are a Mac or Linux user, rename/delete the GMSPython directory in the Gams root directory.
Cheers
Renger