Building ASCEND for 64-bit Windows: Difference between revisions

From ASCEND
Jump to navigation Jump to search
No edit summary
 
(33 intermediate revisions by the same user not shown)
Line 1: Line 1:
We now have a working version of ASCEND on 64-bit Windows. Before you can use these instructions, you need to carefully follow the rather extensive list if instructions for [[setting up a MinGW-w64 build environment]].  
We now have a working version of ASCEND on 64-bit Windows. Before you can use these instructions, you need to carefully follow the rather long list of instructions for [[setting up a MinGW-w64 build environment]]. If you're interested in building ASCEND for other platforms, see [[building ASCEND]].


== CUnit testing library ==
== Compile-time prerequisites ==


Note the directions for building and installing CUnit as per [[Developer's Manual#CUnit testsuites]]. For MinGW-w64 use the following steps:
=== CUnit testing library ===
 
Note the directions for building and installing CUnit as per [[Developer's Manual#CUnit test suites]]. For MinGW-w64 use the following steps:
   
   
  cd ~
  cd ~
Line 9: Line 11:
  cd cunit
  cd cunit
  ./configure --prefix=/mingw --enable-examples --enable-debug
  ./configure --prefix=/mingw --enable-examples --enable-debug
  make -j4
  make
  make install
  make install


Line 21: Line 23:
You may be able to check the current build status on our [http://build.ascend4.org buildbot].
You may be able to check the current build status on our [http://build.ascend4.org buildbot].


---
=== SUNDIALS ===
{{outdated}}


* now run 'scons -j4'
SUNDIALS is used by the [[IDA]] solver in ASCEND. First download and save sundials-2.4.0 from  the [http://computation.llnl.gov/casc/sundials/download/download.html SUNDIALS download page]. Unpack, build and install as follows:


Things to check:
tar zxf /c/Users/'''yourusername'''/Downloads/sundial-2.4.0.tar.gz
* if you already have MinGW on your system the above instructions will need tweaking. We have only tried this on a system without Python or MinGW already installed.
cd sundials-2.4.0
* be sure you have the 64-bit Python package.
./configure --build=x86_64-w64-mingw32 --prefix=/mingw F77=gfortran
* the above will build just 'libascend'. For the CUnit test suite, other solvers, and the GUI, please read on.
make
make install


We still want to check/look into:
SUNDIALS at v2.4.0 doesn't support <tt>DESTDIR=~/temp-install</tt> in the <tt>make install</tt> step, which is annoying becuase it's not quite as easy to 'catch' the installed files and distribute a binary.
* can we set up a MinGW/MinGW-w64 combined build environment and build both 32- and 64-bit versions on that same machine? Can Python 32-bit and 64-bit coexist? Can we install 32- and 64-bit versions of all the Python modules, and GTK as well?
* rather than using the big bundle MSYS that is provided by the MinGW-w64 project, we should try to use 'standard' MinGW-get-inst for the MSYS portion, and load MinGW-w64 on top of that. Makes obtaining minor utilities like 'find' and 'autoconf' much easier.
* our build tools can be 32-bit, even if the build target is 64-bit.


== Test suite ==
Note, for 32-bit, use <tt>./configure --prefix=/mingw F77=gfortran</tt> above. We need to test again whether the <tt>--build=x86_64-w64-mingw32</tt> part is needed for 64-bit builds, or not.


You can now build the ASCEND Python module, <tt>ascpy</tt>. It seems to work, but as of writing, there is a crash in the GUI when instantiating a model. It might be that there are still some errors in relation with 64-bit data types, possibly in the {{srcdir|ascxx}} code. With the above, <tt>./test.py TestSolver.testsunpos6</tt> loads and solves a model correctly! Python bindings at least partly operational! Also, <tt>pygtk/ascdev models/test/ipopt/test6.a4c</tt> seems to work too, if the runtime dependencies listed below are first installed.
=== IPOPT ===


=== Notes about Python with MinGW-w64 ===
[[IPOPT]] is a free open-source optimisation solver which can be accessed from ASCEND.


There seem to be warnings on the net about MinGW and MinGW-w64 used with SWIG. It has always worked fine for us on Win32, but maybe there are new issues arising. To test this, here is a simple 'hello world' example SWIG module. It works on Ubuntu 11.10 32-bit, and builds fine on Win7 64-bit, but fails with a segfault when exiting.
* Download and unpack [http://www.coin-or.org/download/source/Ipopt/Ipopt-3.11.3.tgz Ipopt-3.11.3.tgz]:
 
cd ~
'''[[Media:swigtest.tar.gz|swigtest.tar.gz]]''' (900 byte tarball)
wget http://www.coin-or.org/download/source/Ipopt/Ipopt-3.11.3.tgz
 
tar zxf Ipopt-3.11.3.tgz
MinGW-w64 can build (without SWIG) against Python 2.7 for at least trivial bindings... we can prove it. A simple example that takes a string, prints it, and calculates its length, seems to work fine:
cd Ipopt-3.11.3
* Download the third-party dependencies
cd ThirdParty/Blas && ./get.Blas
cd ../Metis && ./get.Metis
cd ../Mumps && ./get.Mumps
cd ../Lapack && ./get.Lapack
* Configure and build IPOPT (configuration takes a several minutes, building takes ~15 min or so):
cd ~/Ipopt-3.11.3
./configure --enable-static=yes --enable-shared=no --prefix=/mingw ADD_FFLAGS="-fopenmp -static-libgcc" --disable-pthread-mumps
make
make install
make test


'''[[Media:pytest.tar.gz|pytest.tar.gz]]''' (1.1k tarball)
Note: above <tt>./configure</tt> command is a special workaround for 64-bit Windows. Thanks for Stefan Vigerske and Tony Kelman for their help with these instructions (and for [http://www.mpclab.net/Trac/wiki/CompilingIpopt#MinGW this page]), and for the suggested changes to our [[Setting up a MinGW-w64 build environment|MinGW-w64 set-up guidelines]]. See also [https://projects.coin-or.org/Ipopt/ticket/215 IPOPT bug 215]. It is probable that on 32-bit, a simpler command without the 'ADD_FFLAGS' will work OK, but it may change the resulting DLL dependencies of the final DLL/executables.
 
Both of the above tests work fine if the patch from [http://bugs.python.org/issue4709 python bug 4709] is applied to your installed Python pyconfig.h header file.
 
Other pages on MinGW with Python:
* http://sebsauvage.net/python/mingw.html (32-bit version)
* http://boodebr.org/main/python/build-windows-extensions
 
== Cunit ==
 
* use SVN code from the branch 'mingw64':
** cd
** svn co https://cunit.svn.sourceforge.net/svnroot/cunit/branches/mingw64 cunit
** cd cunit
* <tt>./configure --prefix=/mingw --host=x86_64-w64-mingw32 --build=mingw32</tt>
* make install
 
== SUNDIALS ==
 
SUNDIALS is used by the [[IDA]] solver in ASCEND.
 
* <tt>./configure --host=x86_64-w64-mingw32 --prefix=/mingw</tt>
* <tt>make -j4</tt>
* <tt>make install</tt>
 
== IPOPT ==
 
* download and extract the 'findutils' lzma tarball from [http://sourceforge.net/projects/mingw/files/MSYS/Base/findutils/findutils-4.4.2-2/findutils-4.4.2-2-msys-1.0.13-bin.tar.lzma/download here]; put the extract 'find.exe' and 'xargs.exe' into c:/msys/bin.
* download the source tarball for IPOPT 3.10.1
* <tt>cd ThirdParty/Blas && ./get.Blas </tt> (note that on my system I had to edit the URL in the <tt>get.Blas</tt> script to use "http:" instead of "ftp:")
* <tt>cd ThirdParty/Metis && ./get.Metis</tt>
* <tt>cd ThirdParty/Mumps && ./get.Mumps</tt>
* <tt>cd ThirdParty/Lapack && ./get.Lapack</tt> (again I needed to edit the script to switch 'ftp' to 'http'.
* <tt>./configure --enable-static=yes --enable-shared=no --host=x86_64-w64-mingw32 --prefix=/mingw</tt>
* <tt>make -j4</tt>
* <tt>make install</tt>


If the install works OK you should have some /mingw/libcoin* and /mingw/libipopt.a files.
If the install works OK you should have some /mingw/libcoin* and /mingw/libipopt.a files.


This will build you a static IPOPT solver. There isn't a way to build a DLL of IPOPT with MinGW yet, apparently.
This will build you a static IPOPT solver. There isn't a way to build a DLL of IPOPT with MinGW yet, apparently (although the static libary for IPOPT can be incorporated into a DLL as we have done for our [[IPOPT]] external solver in ASCEND).


To get <tt>scons</tt> to detect this build of IPOPT, use
Currently the resulting IPOPT depends on libgfortran and libstdc++. If you build the resulting static libraries into a DLL or an EXE, that DLL/EXE will have shared library dependencies on the ligfortran and libstdc++ DLLs that come with MinGW/MinGW-w64. We would rather like to have the code from of those external DLLs statically linked, but don't yet know how that's possible.


scons --config=force -j4 IPOPT_PREFIX=/mingw IPOPT_LIBS=ipopt,coinmumps,coinmetis,coinlapack,coinblas,gfortran,stdc++
== Building ASCEND ==


The above seems to work fine, except that it results (on my system) in dynamic link to <tt>libgfortran-3.dll</tt>, <tt>libquadmath-0.dll</tt>, <tt>libstdc++-6.dll</tt> and <tt>libgcc_s_sjlj0.dll</tt> all of which must be satisfied at runtime in order for IPOPT to work. We can (and have succeeded to) include those dependencies with the ASCEND installer, but they total an additional 12 MB which we would rather avoid.
If your [[Setting up a MinGW-w64 build environment|Windows build environment]] is set up correctly, and you have a current copy of the source code (either from [[VersionManagement|our subversion repository]] or a source-code tarball), you should be able to simply run:


To build the ASCEND installer in that case, we use
scons -j4


scons IPOPT_DLL1=... IPOPT_DLL2=.... IPOPT_DLL3=... IPOPT_DLL4=... installer
If your environment is set up correctly, your build should complete correctly on both 64-bit and 32-bit build environments.


where the '...' correspond to the full path to the required DLLs on your particular system. On mine they were in /mingw/lib/i686-w64-mingw32/lib or something like that.
To run test suites, try


We would like to remove the need for these extra DLLs. One approach would be to link statically to them, but it doesn't seem to work. We tried building IPOPT 3.10.2 with
scons test ascend solvers models
test/test_expectpass


  ./configure --enable-shared=no --enable-static=yes FFLAGS="-static-libgfortran" CFLAFGS="-static-libgcc" CXXFLAGS="-static-libstdc++ -static-libgcc" --host=x86_64-w64-mingw32 --prefix=/mingw
This script should run all the test cases currently expected to pass. You can also try


but even with all that stuff, the links were still present when it came to creating the ascend_ipopt.dll file.
scons test ascend solvers models MALLOC_DEBUG=1
test/test_expectpass_mallocdebug


== GDB ==
to test those functions which require malloc debugging to also be tested.  This latter set of tests removes some other ones because of known memory leakage.


* Download the GDB package from the MinGW-64 site [https://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/gdb/ here] (we chose [http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/gdb/x86_64-w64-mingw32-gdb-7.1.90.20100730.zip/download x86_64-w64-mingw32-gdb-7.1.90.20100730.zip])
== Runtime prerequisites ==
* Building GDB from source tarball version 7.3.1 worked OK but the resulting GDB didn't recognise/load symbols from the running executable. So download the MinGW-64 pre-compiled version instead.
 
== Runtime requirements ==


To actually run the GUI resulting from the above build, you still need to install GTK+, PyGTK, PyCairo, PyGObject. Get the amd64 py2.7 packages from this page:
To actually run the GUI resulting from the above build, you still need to install GTK+, PyGTK, PyCairo, PyGObject. Get the amd64 py2.7 packages from this page:


* http://ftp.gnome.org/pub/GNOME/binaries/win64/gtk+/2.22/
* Install GTK+2 as detailed in [[setting up a MinGW-w64 build environment]].
** get the GTK+ 2.22 runtime ''''bundle''''
* From the [http://sourceforge.net/projects/ascend-sim/files/thirdparty/ ASCEND third-party download page], download and install:
** extract to c:\GTK
** [http://sourceforge.net/projects/ascend-sim/files/thirdparty/pygtk-2.22.0.win-amd64-py2.7.exe/download pygtk-2.22.0.win-amd64-py2.7.exe]
** make sure you don't have any other GTK installations on your path
** [http://sourceforge.net/projects/ascend-sim/files/thirdparty/pygobject-2.28.6.win-amd64-py2.7.exe/download pygobject-2.28.6.win-amd64-py2.7.exe]
 
** [http://sourceforge.net/projects/ascend-sim/files/thirdparty/py2cairo-1.10.0.win-amd64-py2.7.exe/download py2cairo-1.10.0.win-amd64-py2.7.exe]
* http://www.lfd.uci.edu/~gohlke/pythonlibs
* In order to have plotting and console support in the PyGTK GUI, Download and install the following Python packages from http://www.lfd.uci.edu/~gohlke/pythonlibs (sorry, this website doesn't allow direct URL links). Be sure to use the'''win-amd64-py2.7''' versions for 64-bit Python, or the '''win32-py2.7''' files for 32-bit Python:
** <tt>py2cairo-1.10.0.win-amd64-py2.7.‌exe</tt>
** six (required for dateutil)
** <tt>pygobject-2.28.6.win-amd64-py2.7.‌exe</tt>
** pyparsing (required for matplotlib)
** <tt>pygtk-2.22.0.win-amd64-py2.7.‌exe</tt>
** dateutil (required for matplotlib)
** numpy-MKL (required for matplotlib)
** matplotlib (optional)
** setuptools (required for ipython)
** pyreadline (required for ipython)
** ipython (optional)


To test the above:
To test the above:
* c:\GTK\bin must be in your PATH when attempting to run anything that uses GTK.
* You should be able to start IPython (pylab mode) from the Start menu, without any errors output
* gtk-demo.exe should work and pop up a window with some GUI demos
* c:\Program Files\GTK+-2.22\bin\gtk-demo.exe should run from the windows command prompt, and pop up a window with some GUI demos
* start up Python and try
* After setting your PATH to include <tt>c:\Program Files\GTK+-2.22\bin</tt> (and possibly log out and log in again) you should be able to start Python and try:
<source lang=py>import gtk
<source lang=py>import gtk
w = gtk.Window()
w = gtk.Window()
Line 139: Line 118:
* http://gramps-project.org/wiki/index.php?title=ImportError:_DLL_load_failed
* http://gramps-project.org/wiki/index.php?title=ImportError:_DLL_load_failed


Some optional components that will allow the ASCEND GUI to do more:
== Building the ASCEND installer ==
* http://www.lfd.uci.edu/~gohlke/pythonlibs/
** IPython
** NumPy
** Matplotlib
 
== Creating the installer ==


If you have NSIS installed on your system then you will be able to build working installer for ASCEND. It's a 32-bit installer that installs a 64-bit ASCEND library/executable/etc.
If you have NSIS installed on your system then you will be able to build working installer for ASCEND. It's a 32-bit installer that installs a 64-bit ASCEND library/executable/etc.
Line 158: Line 131:
* what happens when you try to install on Windows 32? does it fail gracefully?
* what happens when you try to install on Windows 32? does it fail gracefully?
* consider including all required dependencies in the package: total size would be +~45MB (minus compression savings, minus optimisations from removing unneeded bits of GTK+?)
* consider including all required dependencies in the package: total size would be +~45MB (minus compression savings, minus optimisations from removing unneeded bits of GTK+?)
[[Category:Development]]

Latest revision as of 07:42, 5 December 2013

We now have a working version of ASCEND on 64-bit Windows. Before you can use these instructions, you need to carefully follow the rather long list of instructions for setting up a MinGW-w64 build environment. If you're interested in building ASCEND for other platforms, see building ASCEND.

Compile-time prerequisites

CUnit testing library

Note the directions for building and installing CUnit as per Developer's Manual#CUnit test suites. For MinGW-w64 use the following steps:

cd ~
svn co svn://svn.code.sf.net/p/cunit/code/branches/mingw64 cunit
cd cunit
./configure --prefix=/mingw --enable-examples --enable-debug
make
make install

The CUnit test suites seem to be basically working. We have made changes to CUnit and currently require that you access the svn branch 'mingw64' version of CUnit for ASCEND testing. A new CUnit release is planned.

Currently, failing test cases on MinGW-w64 are:

    • compiler_autodiff (issue with a non-null pointer in line 198)
    • compiler_bintok (program hangs! possibly just a configuration issue? looks like the parser is waiting for input from stdin)
    • compiler_blackbox (lacking error code in return from parse. not Win64 specific)

You may be able to check the current build status on our buildbot.

SUNDIALS

SUNDIALS is used by the IDA solver in ASCEND. First download and save sundials-2.4.0 from the SUNDIALS download page. Unpack, build and install as follows:

tar zxf /c/Users/yourusername/Downloads/sundial-2.4.0.tar.gz
cd sundials-2.4.0
./configure --build=x86_64-w64-mingw32 --prefix=/mingw F77=gfortran
make
make install

SUNDIALS at v2.4.0 doesn't support DESTDIR=~/temp-install in the make install step, which is annoying becuase it's not quite as easy to 'catch' the installed files and distribute a binary.

Note, for 32-bit, use ./configure --prefix=/mingw F77=gfortran above. We need to test again whether the --build=x86_64-w64-mingw32 part is needed for 64-bit builds, or not.

IPOPT

IPOPT is a free open-source optimisation solver which can be accessed from ASCEND.

cd ~
wget http://www.coin-or.org/download/source/Ipopt/Ipopt-3.11.3.tgz 
tar zxf Ipopt-3.11.3.tgz
cd Ipopt-3.11.3
  • Download the third-party dependencies
cd ThirdParty/Blas && ./get.Blas
cd ../Metis && ./get.Metis
cd ../Mumps && ./get.Mumps
cd ../Lapack && ./get.Lapack
  • Configure and build IPOPT (configuration takes a several minutes, building takes ~15 min or so):
cd ~/Ipopt-3.11.3
./configure --enable-static=yes --enable-shared=no --prefix=/mingw ADD_FFLAGS="-fopenmp -static-libgcc" --disable-pthread-mumps
make
make install
make test

Note: above ./configure command is a special workaround for 64-bit Windows. Thanks for Stefan Vigerske and Tony Kelman for their help with these instructions (and for this page), and for the suggested changes to our MinGW-w64 set-up guidelines. See also IPOPT bug 215. It is probable that on 32-bit, a simpler command without the 'ADD_FFLAGS' will work OK, but it may change the resulting DLL dependencies of the final DLL/executables.

If the install works OK you should have some /mingw/libcoin* and /mingw/libipopt.a files.

This will build you a static IPOPT solver. There isn't a way to build a DLL of IPOPT with MinGW yet, apparently (although the static libary for IPOPT can be incorporated into a DLL as we have done for our IPOPT external solver in ASCEND).

Currently the resulting IPOPT depends on libgfortran and libstdc++. If you build the resulting static libraries into a DLL or an EXE, that DLL/EXE will have shared library dependencies on the ligfortran and libstdc++ DLLs that come with MinGW/MinGW-w64. We would rather like to have the code from of those external DLLs statically linked, but don't yet know how that's possible.

Building ASCEND

If your Windows build environment is set up correctly, and you have a current copy of the source code (either from our subversion repository or a source-code tarball), you should be able to simply run:

scons -j4

If your environment is set up correctly, your build should complete correctly on both 64-bit and 32-bit build environments.

To run test suites, try

scons test ascend solvers models
test/test_expectpass

This script should run all the test cases currently expected to pass. You can also try

scons test ascend solvers models MALLOC_DEBUG=1
test/test_expectpass_mallocdebug

to test those functions which require malloc debugging to also be tested. This latter set of tests removes some other ones because of known memory leakage.

Runtime prerequisites

To actually run the GUI resulting from the above build, you still need to install GTK+, PyGTK, PyCairo, PyGObject. Get the amd64 py2.7 packages from this page:

To test the above:

  • You should be able to start IPython (pylab mode) from the Start menu, without any errors output
  • c:\Program Files\GTK+-2.22\bin\gtk-demo.exe should run from the windows command prompt, and pop up a window with some GUI demos
  • After setting your PATH to include c:\Program Files\GTK+-2.22\bin (and possibly log out and log in again) you should be able to start Python and try:

Invalid language.

You need to specify a language like this: <source lang="html">...</source>

Supported languages for syntax highlighting:

a4c, abap, abc, abnf, actionscript, ada, agda, alan, algol, ampl, amtrix, applescript, arc, arm, as400cl, ascend, asciidoc, asp, aspect, assembler, ats, autohotkey, autoit, avenue, awk, ballerina, bat, bbcode, bcpl, bibtex, biferno, bison, blitzbasic, bms, bnf, boo, c, carbon, ceylon, charmm, chill, chpl, clean, clearbasic, clipper, clojure, clp, cmake, cobol, coffeescript, coldfusion, conf, cpp2, critic, crk, crystal, cs_block_regex, csharp, css, d, dart, delphi, diff, dockerfile, dts, dylan, ebnf, ebnf2, eiffel, elixir, elm, email, erb, erlang, euphoria, exapunks, excel, express, factor, fame, fasm, felix, fish, fortran77, fortran90, frink, fsharp, fstab, fx, gambas, gdb, gdscript, go, graphviz, haml, hare, haskell, haxe, hcl, html, httpd, hugo, icon, idl, idlang, inc_luatex, informix, ini, innosetup, interlis, io, jam, jasmin, java, javascript, js_regex, json, jsp, jsx, julia, kotlin, ldif, less, lhs, lilypond, limbo, lindenscript, lisp, logtalk, lotos, lotus, lua, luban, makefile, maple, markdown, matlab, maya, mercury, meson, miranda, mod2, mod3, modelica, moon, ms, msl, mssql, mxml, n3, nasal, nbc, nemerle, netrexx, nginx, nice, nim, nix, nsis, nxc, oberon, objc, ocaml, octave, oorexx, org, os, oz, paradox, pas, pdf, perl, php, pike, pl1, plperl, plpython, pltcl, po, polygen, pony, pov, powershell, pro, progress, ps, psl, pure, purebasic, purescript, pyrex, python, q, qmake, qml, qu, r, rebol, rego, rexx, rnc, rpg, rpl, rst, ruby, rust, s, sam, sas, scad, scala, scilab, scss, shellscript, slim, small, smalltalk, sml, snmp, snobol, solidity, spec, spn, sql, squirrel, styl, svg, swift, sybase, tcl, tcsh, terraform, tex, toml, tsql, tsx, ttcn3, txt, typescript, upc, vala, vb, verilog, vhd, vimscript, vue, wat, whiley, wren, xml, xpp, yaiff, yaml, yaml_ansible, yang, zig, znn

Some helpful diagnostics for finding out why 'import gtk' gives errors in some cases: (in my case, I needed to add c:/GTK/bin to the START of my PATH because of a conflicting ZLIB1.DLL earlier in my PATH.

Building the ASCEND installer

If you have NSIS installed on your system then you will be able to build working installer for ASCEND. It's a 32-bit installer that installs a 64-bit ASCEND library/executable/etc.

Before you can build the installer, you need to download the NSIS Inetc plugin, and extract the inetc.dll file into your c:\Program Files (x86)\NSIS plugins folder.

Some issues to address/check:

  • make sure no WoW3264Node confusions arise when trying to install 64-bit ASCEND on 64-bit Windows.
  • possible issue with IPOPT with 32-bit installer used on 64-bit Windows. Also check IPOPT with 64-bit version.
  • how to make sure that installed versions of GTK/Python etc are 64 bit and not 32-bit?
  • are there any issues with MSVCR90.DLL not being present? PyGTK seems to depend on it.
  • what happens when you try to install on Windows 32? does it fail gracefully?
  • consider including all required dependencies in the package: total size would be +~45MB (minus compression savings, minus optimisations from removing unneeded bits of GTK+?)