Guile (2.0.13) is incompatible with meep: Temporary workaround

[JohnWeiner]

It appears that the latest version of Guile (2.0.13) is incompatible with meep. The first meep tutorial (straight waveguide) does not run with Guile 2.0.13 installed. Here is a summary of what I did to get it to work (in single processor mode)

1. Remove Guile 2.0.13 using apt remove
2. Download deb packages for Guile 2.0.11, Guile-dev 2.0.11, Guile-libs 2.0.11
3. Use dpkg -i for installation in the following sequence: (a) Guile-lib, (b) Guile, (c) Guile-dev
4. Use the python-meep-install.sh script, but edit it so as to choose the “serial” option and comment out the line that invokes installation of guile.

I found that the install script runs to completion. Even though the meep mode is “serial” (single processor, not multiprocessor) the command to run meep is still meep-mpi.
A check of the guile version shows 2.0.11 installed.

The first tutorial on the Meep tutorial web page (straight wave guide) ran without error and generated correct h5 files. The uses h5utils to convert the h5 files to .png files ran without error.

[FilipDominec]

Thank you, John. This appears as a fundamental incompatibility which should be fixed upstream, i.e. by actual MEEP developers, and they should be aware of it now.

If nothing changes and MEEP does not compile in new Ubuntu until September 2017, I will take some measures to explicitly download the legacy libraries.

[JohnWeiner]

Hi Filip, I think that Ardavan is aware of the problem…he seems to think it has to do with SWIG that has failed to take account of a change that Guile made between versions 2.0.11 and 2.0.12. In fact the installation of meep-1.3 is not too difficult on the Ubuntu box. The meep package does have all the dependencies, but I don’t think there is any way to tell the package handler, apt or apt-get, to install guile-2.0.11 instead of the latest version. Here is essentially what I did to get around the problem: First, I installed guile-2.0.11 on /usr/local from downloading the tarball and building it from sources. I will send the list of dependencies I needed if and when you want them, but I think most, if not all, of them are already in the meep package. I then modified the $PATH to make sure that /usr/local/bin is searched BEFORE /usr/bin. It might be that a more recent guile version exists (either from the Ubuntu distro or the meep package) in /usr/bin, /usr/lib, /usr/include, but if the search path starts with /usr/local/bin, then the guile-2.0.11 will be invoked. Second, there is a real bug with a file called stime.h that is in …/libguile. The version that comes with guile 2.0.11 does not work, but all you have to do is copy stime.h from a later version into the libguile that is installed at the /usr/local level. With those two fixes everything else should be ok on Ubuntu. On the macOS I had a lot problems because the dependencies are not well-documented and one just finds what is missing by trial and error. Now everything is running well on both systems and I can get back to substantive, scientific problems. Best regards, John

…

On 1 Aug 2017, at 22:47, Filip Dominec ***@***.***> wrote: Thank you, John, and thank you again for posting more elaborate guide in the mailing list <http://meep-discuss.ab-initio.mit.narkive.com/3sHyPdvB/notes-on-installing-guile-meep-on-macos-sierra-version-3-july-2017>. This appears as a fundamental incompatibility which should be fixed upstream, i.e. by actual MEEP developers, and they should be aware of it now. If nothing changes and MEEP does not compile in new Ubuntu until September 2017, I will take some measures to explicitly download the legacy libraries. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHg6eBDnx2rtepLMoUzp5uSn9IeBmV3qks5sT47fgaJpZM4N9eXD>.

[FilipDominec]

John (and all folks bitten by the update of Guile and obsoleteness of Swig),
as I promised, I added a workaround that detects that package Guile1.8 is not available, and that Guile2.0 is also not available in its version of 2.0.11 or older.

In such a case, it downloads the Guile git source, and checkouts to its old-enough version. Thus, the meep compilation seems to be fine on Ubuntu 17.04, as suggested by the latest log https://github.com/FilipDominec/python-meep-install/blob/master/logs/171006-ubuntu17.04-amd64-02withguile11fromgit.log

Note that there is another compatibility issue in installation on new Ubuntu versions, as reported here: #10

I hope that the described update spares you some time.

[FilipDominec]

As I think about it - perhaps your approach of cleanly installing Guile into the /usr/local/bin is preferable. Could you please post the code that does the job here?

So far, I probably did not encounter the bug with stime.h. Could you please describe the situation when it becomes a trouble?

[JohnWeiner]

Hi Filip, Thanks for the new script. I browsed the log file looking at the various warnings. Some of them look familiar, but as long as warnings don’t turn into errors the installation continues. I did note on the very last line of the log file, error: command 'x86_64-linux-gnu-gcc' failed with exit status 1. Is this as ominous as it sounds? I haven’t had the courage to try python-meep or mpi-meep yet. I have a working scheme-meep system that runs on a single core. For long parameter sweeps, it takes longer than a parallel version would of course, but at least I’ve been able to work on some physics questions and take a break from system/development problems. My next installation project will be to get mpi-meep working. I have a motherboard with 16 cores in the cpu. My hope is that they will work in parallel once the mpi version is installed. With respect to installing guile-2.0.11 in /usr/local/bin, I just downloaded the pertinent tarball from the gnu site and went through the usual ./config make make install sequence. One has to pay careful attention to dependencies of course. I think that either the README or INSTALL files that come with the tarball has a list of the needed dependencies. My experience has been that using the package handlers like home-brew, MacPorts, or apt leads to problems. They download and install the latest versions of everything, and sometimes that leads to problems with guile-2.0.11. All of this will go away, I presume, once swig is updated…don’t know who maintains swig or even if they are aware of the problem. Best regards, John

…

On 5 Oct 2017, at 23:51, Filip Dominec ***@***.***> wrote: —

[FilipDominec]

Hi, John, thanks for your message.
As you probably assume,

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1.

is actually fatal for python-meep compilation. It however means that something failed in the whole compilation chain, and one has to find out what. I spent about two hours with this investigation, and eventually I found out that the recent git version of the script compiles correctly under Ubuntu 17.04 (amd64) with Guile 2.0.13 taken from the repository. This is puzzling, since I thought this is exactly the version (both of Guile and of Ubuntu) when the compilation broke.

Regards, Filip

[JohnWeiner]

Hi Filip It used to be that all versions of Guile later than 2.0.11 would not RUN correctly, even when they compiled and linked to meep correctly. What I had found out was that, when running the first waveguide tutorial, I would get a run time error that failed to produce usable output. Furthermore, the h5 utils, especially h5topng, threw segmentation faults. Ardavan Oskooi surmised that it had to do with SWIG not adjusting to a Guile change after 2.0.11, but I have no independent judgment about that. Maybe SWIG has caught up with Guile. There have been a few messages, one from SJ, on the meep-discuss about later versions of guile now working, but other messages contradicting this good news. I don’t know what the status of things is right now. I do know that I have a working meep environment using guile-2.0.11 running on an iMac, a MacBook, and Ubuntu 17.04 (amd64). As far as I know Guile-2.0.11 always works (for the time being), although some aspects of it have apparently been deprecated and may eventually disappear. Best regards, John

…

On 8 Oct 2017, at 23:26, Filip Dominec ***@***.***> wrote: Hi, John, thanks for your message. As you probably assume, error: command 'x86_64-linux-gnu-gcc' failed with exit status 1. is actually fatal for python-meep compilation. It however means that something failed in the whole compilation chain, and one has to find out what. I spent about two hours with this investigation, and eventually I found out that the recent git version of the script compiles correctly under Ubuntu 17.04 (amd64) with Guile 2.0.13 taken from the repository. This is puzzling, since I thought this is exactly the version (both of Guile and of Ubuntu) when the compilation broke. Regards, Filip — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHg6ePPBsG00-jQ4qapfJ7aSFN5_CpS0ks5sqT3pgaJpZM4N9eXD>.

[FilipDominec]

Hi, I can positively state that the installer script and both the simulations finish successfully with Guile 2.0.13 installed from the Ubuntu-17.04 repository. So it seems that the original issue somehow magically disappeared.

[JohnWeiner]

Try it with Guile 2.2.2…I think that is the latest. John

…

On 9 Oct 2017, at 10:02 , Filip Dominec ***@***.***> wrote: Hi, I can positively state that the installer script and both the simulations finish successfully with Guile 2.0.13 installed from the Ubuntu-17.04 repository. So it seems that the original issue somehow magically disappeared. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHg6eJG61aINnfr7Lfvu7rpUJJWDw9KEks5sqdMTgaJpZM4N9eXD>.

[JohnWeiner]

Hi again Filip I think guile 2.2 is the version packaged with homebrew and apt-get. If it works with 2.2 then meep can be installed from the binary packages which simplifies life for everybody. John Envoyé de mon iPhone

…

Le 9 oct. 2017 à 10:02, Filip Dominec ***@***.***> a écrit : Hi, I can positively state that the installer script and both the simulations finish successfully with Guile 2.0.13 installed from the Ubuntu-17.04 repository. So it seems that the original issue somehow magically disappeared. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[FilipDominec]

Hi, as I already mentioned in my recent post under #10, I made a new test over different Ubuntu versions. In short, the script finishes correctly in all of them without the need to manually compile Guile. I do not know what changed, but it just works.

Here I post the versions of Guile found in each system.

Ubuntu version     "guile-1.8-dev"      "guile-2.0-dev"      Compilation result
--------------     ---------------      ---------------      ------------------
16.04 "trusty"	   1.8.8+1-10ubuntu1    2.0.11+1-10          OK with 1.8
16.04.3 "trusty"   1.8.8+1-10ubuntu1    2.0.11+1-10          OK with 1.8
16.10 "yakkety"    1.8.8+1-10ubuntu1    2.0.11+1-12build2    OK (both with 1.8 and 2.0)
17.04 "zesty"      (N/A)                2.0.13+1-4           OK (now) with 2.0
17.10 "artful"     (N/A)                2.0.13+1-4           OK (as in "zesty")

Note that no package like "guile-2.2-dev" exists in any official repo (yet).