Setting up a MinGW-w64 build environment

From ASCEND
Jump to: navigation, search

This page documents a suggested C/C++/Fortran/Python software build environment for Windows, tested on Windows 7 Professional 64-bit. The build environment is capable of building both 64-bit and 32-bit software, although some manual editing of configuration files is required to switch between the two. With some fairly obvious changes, these instructions will also work for building 32-bit software using Windows XP 32-bit.

The suggested build environment includes

  • 7-zip file archive tool
  • Python 2.7.5 scripting language
  • MSYS, from the official MinGW project
  • MinGW-w64 GCC 4.7.1 compiler suite, from TDM-GCC
  • SCons 2.3.0 software build tool
  • gedit 2.30.1 text editor
  • SWIG 2.0.10 wrapper interface generator
  • gendef (mingw-w64 svn trunk, revision 5774) linker .def file generator
  • NSIS installation package builder
  • Dependency Walker 2.2 library/program file dependency checker
  • Subversion 1.8.0 version control software

These steps were completed by a user with administrator privileges. It is known that the Python, MinGW etc can be installed by non-administrator users, but we haven't yet tested that scenario, unfortunately. Problems may potentially arise in relation to 7-zip, gedit, NSIS or Subversion if administrator privileges are not available (but that's just guessing).

It's worth mentioning here that the 64-bit architecture for Windows that we're talking about here is alternatively referred to as x64, x64_64 and amd64. These terms all seem to be used a bit interchangeably.

Contents

First steps

  • Download and install the 64-bit version of 7-zip. We used the 64-bit version, 7z920-x64.msi.
  • Download and install the latest Python 2.7 release. We used the 64-bit version if 2.7.5, python-2.7.5.amd64.msi from this page. Select to install for all users, and select the default installation location, c:\Python27 (although you can use another location if you already have 32-bit Python installed, but be careful to change PATH references further below if necessary).
    • Note: it is permissible to install both 32-bit and 64-bit Python in the same system, and it works fine, if you're careful about setting you PATH. If you also want to build against 32-bit Python, we suggest you download python-2.7.5.msi and install it in c:\Python27_32.
  • Download and install the gedit text editor for Windows. We used this package: gedit-setup-2.30.1-1.exe (32-bit).
    • To ensure that Windows knows that gedit is a text editor, use Windows Explorer to navigate to c:\Program Files (x86)\gedit\share\doc and right-click on README, and select Open. In the dialog, click Browse, then select c:\Program Files (x86)\gedit\bin\gedit.exe, and click OK to open the file. This will ensure that gedit appears in the list of editors for other text files you want to open in future.
    • Note: be aware that some other installers may modify your system PATH in a way that causes gedit to break. If you start to get errors when launching gedit, this is probably what has happened.
  • Download and install NSIS. We used nsis-2.46-setup.exe. You can tell the installer not to actually run NSIS, no need for that. If you will be building ASCEND, you should also download the Inetc plugin and put the DLL from that zip file into the Plugins subdirectory where you installed NSIS.

MSYS

MinGW is a minimal GNU/GCC environment for Windows. MSYS is a command-line environment based on MinGW that mimics the Linux environment sufficiently well to allow a large portion of Linux-compatible software to also be compilable under Windows. There are many different combinations of ways to set up this environment. And things are made quite a bit more complicated by the fact that the official MinGW project does not support the 64-bit fork MinGW-w64, and as yet (Aug 2013) there is no nicely-integrated environment that supports installing everything automatically. Based on our experience in particular with difficulties in building recent versions of IPOPT, this is therefore our current preferred approach:

  • Download mingw-get-setup.exe from the official MinGW (32-bit) project page (we used version 0.6.2-beta-20131004-1), and run it:
    • Select to install MinGW to the directory c:\MinGW\32 (this is not the default location)
    • Select not to install the graphical user interface
Error creating thumbnail: convert: unable to open image `/srv/ascend/d/up/d/df/Mingw-get-setup-0.6b.png': No such file or directory @ error/blob.c/OpenBlob/2638.
convert: unable to open file `/srv/ascend/d/up/d/df/Mingw-get-setup-0.6b.png' @ error/png.c/ReadPNGImage/3667.
convert: no images defined `/tmp/transform_13c8d7-1.png' @ error/convert.c/ConvertImageCommand/3044.
Screenshot of MinGW-get-setup showing selected installation options
  • Open gedit and edit the file c:\mingw\32\var\lib\mingw-get\data\profile.xml and replace the entire contents of the file with the following text:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<profile project="MinGW" application="mingw-get">
  <repository uri="http://prdownloads.sourceforge.net/mingw/%F.xml.lzma?download"/>
  <system-map id="default">
    <sysroot subsystem="mingw32" path="%R" />
    <sysroot subsystem="MSYS" path="%R/../msys" />
  </system-map>
</profile>
c:\mingw\32\bin\mingw-get install msys-core msys-base msys-vim msys-wget msys-patch msys-flex msys-bison
  • Using gedit, create the file c:\mingw\msys\etc\fstab with the following content:
c:/mingw/32   /mingw
  • Using Windows Explorer, create a shortcut to the file c:\mingw\msys\msys.bat and place the shortcut on the desktop. You may wish to set the icon of the shortcut to the icon file c:\mingw\msys\msys.ico.
  • Double-click your new desktop shortcut to msys.bat. You should see a console window as follows:

Msys-windows7-test.png

Switchable 32- and 64-bit modes

Our setup will be a bit of a hack of the standard MinGW environment. We use the official MinGW project to provide the MSYS environment, but then we use the MinGW-w64 project (as below) to provide the actual compiler, since we want 64-bit support. Our file structure will be

  • c:\
    • mingw\ (we install everything under this single top-level directory)
      • 64\ (the 64-bit compiler and all our 64-bit compiled-and-installed libraries will live here)
      • 32\ (the mingw-get binary will already be inside here, as well as the 32-bit compiler if we install it later)
      • msys\ (the MSYS environment, and loads of useful non-compiler tools can be installed here, but they shouldn't have any binary dependencies in c:\mingw\64 or c:\mingw\32, or things will get broken!)

In principle, all we need to do to switch between the 64-bit and 32-bit build environments is to edit the text file /etc/fstab (changing '32' to '64' or vice versa), then reload MSYS. In other words, if you want to now switch to 64-bit mode, you would modify /etc/fstab to read

c:/mingw/64   /mingw

After making this change, be sure to close and re-open your MinGW/MSYS command window (re-open it using the MSYS icon that you created on your desktop).

MinGW-w64

MinGW-w64 (Minimalist GNU for Windows, 64-bit) is a fork of the earlier MinGW GCC (GNU Compiler Collection: C, C++ and Fortran compilers and associated tools). MinGW-w-64 was carefully extended to include support for 64-bit Windows. Despite the name, MinGW-w64 both runs on and builds binaries for both 32-bit and 64-bit Windows, but you need to make sure you download the right package according to which platform you are running on, and which platform you want to target.

  • Download the TDM-GCC 'MinGW-w64 sjlj' installer for from this page. We used tdm64-gcc-4.7.1-3.exe.
  • Install the TDM-GCC package into the directory c:\mingw\64, selecting to install all components. Select not to add MinGW-w64 to the PATH.
  • After the installer completes, use the Start menu, right-click on 'Computer', select 'Properties' then 'Advanced Systems Settings' then 'Advanced' tab, click button 'Environment variables'. Under system variables, edit the variable named PATH and from then end of the path, remove c:\mingw\64\bin. We don't want this in the path because it messes with gedit, and will also cause confusion when running 32-bit compilers.
  • Open the MSYS shell (make sure you are in 64-bit mode, as mentioned above) by double-clicking your new desktop shortcut, and type gcc --version. You should see
john@win7_64 ~
$ gcc --version
gcc.exe (tdm64-1) 4.7.1
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  • In gedit, open (or create) the file c:\mingw\msys\home\yourusername\.profile (no file extension). In that file, paste in the following text, and save the resulting file.

#!/bin/bash
# we use the reported architecture of the 'gcc' in our path to 
# determine which Python and other utilities we will be using.
_arch=`gcc -dumpmachine`
#echo "ARCH=$_arch"
if grep "^x86_64-" <<< "$_arch" >/dev/null ; then
	echo "MINGW 64 BIT BUILD ENVIRONMENT"
	_pydir="/c/Python27"
else
	echo "MINGW 32 BIT BUILD ENVIRONMENT"
	_pydir="/c/Python27_32"
fi

export PATH=$PATH:$_pydir:$_pydir/Scripts

# note that mingw-get will still install all its stuff to c:\mingw\32,
# because of the contents of its profile.xml file.
alias mingw-get="/c/mingw/32/bin/mingw-get"

TODO: add instructions for also installing the 32-bit compilers, and instructions for getting things working on a 32-bit Windows system.

Note: the above TDM 64-bit installer does not install a fortran compiler, but a fortran compiler is required for building ASCEND. The alternative appears to be to use the latest tdm-gcc-webdl.exe downloader-installer. Select all options, but select not to install to PATH.

MinGW 32-bit compilers (optional)

For the 32-bit compilers, we will use the standard MinGW distribution. In MSYS set to 32-bit mode (see above), type:

mingw-get install gcc g++ gfortran

The default MinGW 32-bit compilers use win32 threading model and dwarf2 threading.

To test the installed compiler, run

gcc -dumpmachine
gcc -v

And you should see output that starts with mingw32, confirming you have the 32-bit compiler running.

Note, due to current bug 1950 in MinGW (Dec 2013), you may need to downgrade 'msys-core-bin' package to version 1.0.17 to avoid problems with parallel builds, eg make -j4. The necessary command which should probably be run from an standard Windows command prompt, is

cd c:\mingw\32\bin
mingw-get upgrade msys-core-bin=1.0.17-1

SCons

We need to custom-build our own SCons installer from the source code, because (last time we checked), the SCons installer didn't correctly detect 64-bit Python on our system.

  • Download, unpack, build and install SCons. This can be done using the MSYS window in 64-bit mode (see above):
cd ~
wget http://prdownloads.sourceforge.net/scons/scons-2.3.0.tar.gz
tar zxf scons-2.3.0.tar.gz
cd scons-2.3.0
python setup.py bdist_wininst
start dist/scons-2.3.0.win-amd64.exe 
  • Step through the installer process, then return to your MSYS window and type
cp /c/Python27/Scripts/scons.py /c/Python27/Scripts/scons
  • To ensure that you can do parallel builds (using more than one of your processors at a time), also install pywin32, available from Christophe Gohlke's Python extensions page or from the ASCEND third-party downloads page. Use the amd64 version if you selected 64-bit Python above.
  • Test that SCons is working by typing scons --version. You should get a copyright message including text to show that the you have version 2.3.0 of SCons running:

Scons-version-test-mingw.png

Optional: If you also want to use SCons for 32-bit packages, you can install just the downloadable 'Windows' version of SCons directly from the Scons homepage. We used version 2.3.0. If required, select to install to c:\Python27_32 (the location where you installed 32-bit Python). You still need to copy the /c/Python27_32/Scripts/scons.py to /c/Python27_32/Scripts/scons as above, and you should also install pywin32, 32-bit version (from here) to ensure that parallel builds are properly supported.

Dependency Walker

  • Download the 'x64' version of Dependency Walker. We downloaded this file: depends22_x64.zip. If you are also building 32-bit applications, you will also need the i686 version of Dependency Walker.
  • Unpack the .exe and .dll files into the directory c:\mingw\64\bin, for the 64-bit version and/or c:\mingw\32\bin for the 32-bit version. By utting the files here you will automatically get access to the correct version of depends corresponding to the 32- or 64-bit mode you are running via /etc/fstab
  • Test that it's working by typing into MSYS the command depends /mingw/bin/gcc.exe. A window should pop up showing the DLLs that your GCC compiler depends on.

TODO check that two versions of depends.exe are actually required! Perhaps they're not both needed, in which case we could install to c:\mingw\msys\bin.

SWIG

  • Download a pre-compiled 'swigwin' copy of SWIG. We used swigwin-2.0.10.zip.
  • Create a folder c:\mingw\msys\opt using Windows Explorer, then copy the 'swigwin-2.0.10' folder out of the Zip archive and into the new location c:\mingw\msys\opt\swigwin-2.0.10
  • In MSYS, type gedit ~/.profile, then add the SWIG directory to your PATH in your profile, by adding the following line at the bottom of that file:
export PATH=$PATH:/opt/swigwin-2.0.10
  • Close your MSYS window and reopen it, so that the ~/.profile will be re-read.
  • Test that SWIG is accessible by typing swig -version. You should see the version number and some other information output.

As SWIG is a code pre-processor, it doesn't matter whether it's 32-bit or 64-bit, they will be perfectly equivalent. That is the reason that we install SWIG under msys rather than in mingw\32 or mingw\64. Note that SWIG version 2.0.5 was known to cause problems with MinGW-w64.

GTK+2

GTK+ is useful for developing cross-platform graphical user interface (GUI) programs. It works well under Windows, and it is the native GUI toolkit for Ubuntu, Fedora, Mint, and other Linux distributions. It has also been ported to Mac, although getting up and running on that platform is still a bit difficult. The ASCEND project provides some pre-compiled GTK+ installers that make getting started with GTK+ very straightforward.

  • Download gtk+-2.22.1-20101229-x64-a4.exe from the ASCEND 'files' section, under 'thirdparty'.
  • Again edit your ~/.profile file, this time add GTK, but at the end of your PATH. Add this line inside the part of the 'if' statement relating to your 64-bit setup:
export PATH=$PATH:/c/Program\ Files/GTK+-2.22/bin
  • Close your MSYS and re-open it so that the updated ~/.profile is loaded.
  • Test that GTK+ is working by typing either pkg-config gtk+-2.0 --libs or gtk-demo. For the latter, you should see a GUI window pop up that shows off the various GTK+ features.

Mingw-gtk-test.png

Optional: To develop 32-bit applications with GTK+, download the i386 version of GTK+ from the above 'thirdparty' page, and add it to the PATH using the following line in the 32-bit part of your .profile:

export PATH=$PATH:/c/Program\ Files\ \(x86\)/GTK+-2.22/bin

Note / TODO: GTK+3 is now available (and we have been porting ASCEND to GTK+3), however we haven't tested GTK+3 on Windows up to now. There are some binaries available (64-bit and 32-bit) but they need to be repackaged for direct use on Windows, and tested.

Subversion

Subversion is a version control software. ASCEND uses it for hosting our source code, as do many other projects, especially on SourceForge.net and Google Code. SlikSvn is a nice easy-to-install packaging of Subversion software.

  • Download and install the 64-bit version of SlikSVN, Slik-Subversion-1.8.5-x64.msi from the SlikSVN website. You can use the 'typical' settings and default installation location, which is c:\Program Files\SlikSvn
  • Edit your ~/.profile, adding the following line at the end.
export PATH=$PATH:/c/Program\ Files/SlikSvn/bin
  • Close your MSYS and re-open it so that the updated ~/.profile is loaded.
  • Check that subversion is working by typing svn --version.

It's not necessary to install both 32 and 64-bit versions of Subversion, only one is required, and it will work fine whether you're building 32-bit or 64-bit software.

Setup Python for compilation of extensions

Note: This section is only required if you want to build 64-bit Python extensions. No actions are required here if you are going to build 32-bit Python extensions.

Before you can built any Python extensions using MinGW-w64 (which includes via SWIG), some preparation is required. One C header file, pyconfig.h has to be patched, and linker definitions for python27.dll have to be prepared for the compiler. Follow these instructions to sort it out.

First download, build, and install a utility called 'gendef' which builds library *.def files that MinGW-w64 can use when linhing to DLLs. This is because the python27.lib file distributed with Python doesn't work with MinGW-w64. In your MSYS shell, type

svn co  svn://svn.code.sf.net/p/mingw-w64/code/trunk/mingw-w64-tools/gendef -r5774 ~/gendef
cd ~/gendef 
./configure --prefix=/mingw
make -j4 && make install
cd
gendef --help
# should output starting with 'Usage: gendef...'

Now prepare a new python27.def file from the installed copy of python27.dll in c:\windows\system32:

  • Using Windows Explorer, copy the file c:\windows\system32\python27.dll into the folder c:\Python27\Libs
  • Using MSYS, enter the following:
cd /c/Python27/libs
mv python27.lib old-python27.lib
gendef python27.dll

If this works OK, you should have a new file c:\Python27\Libs\python27.def.

Next, patch the c:\Python27\include\pyconfig.h file:

  • Open the file c:\Python27\include\pyconfig.h using gedit
  • Search for the text #ifdef _WIN64, which in our copy of this file was at line 141, and cut out the following three lines:
#ifdef _WIN64
#define MS_WIN64
#endif
  • Search for the text #ifdef _MSC_VER, which in our copy of this file was at line 107. Paste in the cut-out lines, ABOVE the #ifdef _MSC_VER. Alternatively, if you don't want to do this last step, you could just make sure you define MS_WIN64 before you #include "Python.h" in your source code.

This issue with MS_WIN64 is reported as python bug 4709 in the Python bug tracker, so possibly it will have been fixed if you use a newer version of Python than we tested with.

We are finding that the above steps give correct behaviour on Windows 7. If future releases cause problems, the following may be useful for testing:

.profile

At the end of these steps, the .profile file (c:\mingw\msys\home\yourusername\.profile) will contain the following.

#!/bin/bash
# we use the reported architecture of the 'gcc' in our path to 
# determine which Python and other utilities we will be using.
_arch=`gcc -dumpmachine`
#echo "ARCH=$_arch"
if grep "^x86_64-" <<< "$_arch" >/dev/null ; then
	echo "MINGW 64 BIT BUILD ENVIRONMENT"
	_pydir="/c/Python27"
	export PATH=$PATH:/c/Program\ Files/GTK+-2.22/bin/
else
	echo "MINGW 32 BIT BUILD ENVIRONMENT"
	_pydir="/c/Python27_32"
	export PATH=$PATH:/c/Program\ Files\ \(x86\)/GTK+-2.22/bin/
fi

export PATH=$PATH:$_pydir:$_pydir/Scripts

# note that mingw-get will still install all its stuff to c:\mingw\32 and c:\mingw\msys.
# because of the contents of its profile.xml file. it is not affected by the content of /etc/fstab.
alias mingw-get="/c/mingw/32/bin/mingw-get"

export PATH=$PATH:/opt/swigwin-2.0.10
export PATH=$PATH:/c/Program\ Files/SlikSvn/bin

Usage tips

All done! You may wish to make a backup of your directory c:\mingw, which contains your complete MSYS and MinGW setup, for easy reinstallation later. You would still in that case need to install SlikSVN, GTK+, Python, Scons, Gedit, NSIS, 7zip, but all of the manual fiddling about would be avoided.

When building software on this platform using GNU AutoTools, (ie, the typical './configure && make && make install' instructions), you should specify --prefix=/mingw to ensure files are installed in a path that corresponds to the 32- or 64-bit version of your compiler. This will make library linking much easier, at least while you stay within MSYS. Also, you may need to specify your build environment using the --build flag. For example,

./configure --prefix=/mingw --build=x86_64-w64-mingw32
make 
make install

This will have the result that libraries and new tools will be installed into c:\mingw\64.

If you ever want to switch building 32-bit code, recall the instructions above about #Switchable 32- and 64-bit modes. This re-routes "/mingw" to your 64-bit or 32-bit compiler and libraries according to which setting to you choose. Then, the 'if' statement in the .profile file above takes care of replacing 64-bit stuff with the 32-bit alternatives. This approach also has the result that, when running ./configure scripts, there is usually no need to add the --build flag to specify the 32- or 64-bit build architecture, as only one compiler will be visible/accessible at any one time, anyway.

Example of building a simple program

If everything is working, you should be able to enter the following simple C program into gedit, and save it into c:\mingw\msys\home\yourusername\test.c:

#include <stdio.h>
int main(){
  printf("Hello, world!\n");
  return 0;
}

Then compile the file and run it by typing

gcc test.c
./a.exe

And you should see

Hello, world!

TODO: expand with simple MinGW-w64 example using SCons as the build tool.

See also

Personal tools
Namespaces

Variants
Actions
Navigation
tools
Tools