Setting up a MinGW-w64 build environment

From ASCEND
Revision as of 05:18, 17 August 2013 by Jpye (talk | contribs)
Jump to navigation Jump to search

This page documents a suggested C/C++/Fortran/Python software build environment for Windows, tested on Windows 7 Professional 64-bit. The build environment is capable of building both 64-bit and 32-bit software, although some manual editing of configuration files is required to switch between the two. With some fairly obvious changes, these instructions will also work for building 32-bit software using Windows XP 32-bit.

The suggested build environment includes

  • 7-zip file archive tool
  • Python 2.7.5 scripting language
  • MSYS, from the official MinGW project
  • MinGW-w64 GCC 4.7.1 compiler suite, from TDM-GCC
  • SCons 2.3.0 software build tool
  • gedit 2.30.1 text editor
  • SWIG 2.0.10 wrapper interface generator
  • gendef (mingw-w64 svn trunk, revision 5774) linker .def file generator
  • NSIS installation package builder
  • Dependency Walker 2.2 library/program file dependency checker
  • Subversion 1.8.0 version control software

These steps were completed by a user with administrator privileges. It is known that the Python, MinGW etc can be installed by non-administrator users, but we haven't yet tested that scenario, unfortunately. Problems may potentially arise in relation to 7-zip, gedit, NSIS or Subversion if administrator privileges are not available (but that's just guessing).

First steps

  • Download and install the 64-bit version of 7-zip. We used the 64-bit version, 7z920-x64.msi.
  • Download and install the latest Python 2.7 release. We used the 64-bit version if 2.7.5, python-2.7.5.amd64.msi from this page. Select to install for all users, and select the default installation location, c:\Python27 (although you can use another location if you already have 32-bit Python installed, but be careful to change PATH references further below if necessary).
    • Note: it is permissible to install both 32-bit and 64-bit Python in the same system, and it works fine, if you're careful about setting you PATH. We suggest c:\Python27_32 if you want to also install 32-bit Python on 64-bit Windows.
  • Download and install the gedit text editor for Windows. We used this package: gedit-setup-2.30.1-1.exe (32-bit).
    • To ensure that Windows knows that gedit is a text editor, use Windows Explorer to navigate to c:\Program Files (x86)\gedit\share\doc and right-click on README, and select Open. In the dialog, click Browse, then select c:\Program Files (x86)\gedit\bin\gedit.exe, and click OK to open the file. This will ensure that gedit appears in the list of editors for other text files you want to open in future.
  • Download and install NSIS. We used nsis-2.46-setup.exe. You can tell the installer not to run NSIS, no need for that.

MSYS

MinGW is a minimal GNU/GCC environment for Windows. MSYS is a command-line environment based on MinGW that mimics the Linux environment sufficiently well to allow a large portion of Linux-compatible software to also be compilable under Windows. There are many different combinations of ways to set up this environment. And things are made quite a bit more complicated by the fact that the official MinGW project does not support the 64-bit fork MinGW-w64, and as yet (Aug 2013) there is no nicely-integrated environment that supports installing everything automatically. Based on our experience in particular with difficulties in building recent versions of IPOPT, this is therefore our current preferred approach:

  • Download mingw-get-inst-20120426.exe from the official MinGW (32-bit) project page.
    • Select to use pre-packaged catalogs (not to download updates)
    • Select to install MinGW to the directory c:\mingw\32
    • Select to install no start-menu shortcuts
    • Select to install none of the components, just complete the installation without any packages selected.
  • Open gedit and edit the file c:\mingw\32\var\lib\mingw-get\data\profile.xml and replace it with the following text:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<profile project="MinGW" application="mingw-get">
  <repository uri="http://prdownloads.sourceforge.net/mingw/%F.xml.lzma?download"/>
  <system-map id="default">
    <sysroot subsystem="mingw32" path="%R" />
    <sysroot subsystem="MSYS" path="%R/../msys" />
  </system-map>
</profile>

c:\mingw\32\bin\mingw-get install msys-core msys-vim msys-wget msys-patch
  • Using gedit, create the file c:\mingw\msys\etc\fstab with the following content:
c:/mingw/64   /mingw
  • Using Windows Explorer, create a shortcut to the file c:\mingw\msys\msys.bar and place the shortcut on the desktop. You may wish to set the icon of the shortcut to the icon file c:\mingw\msys\msys.ico.
  • Double-click your new desktop shortcut to msys.bat. You should see a console window as follows:

Our setup is a bit of a hack of the standard MinGW environment. We use the official MinGW project to provide the MSYS environment, but then we use the MinGW-w64 project (as below) to provide the actual compiler, since we want 64-bit support. Our file structure will be

  • c:\
    • mingw\ (we install everything under this single top-level directory)
      • 64\ (the 64-bit compiler and all our 64-bit compiled-and-installed libraries will live here)
      • 32\ (the mingw-get binary will already be inside here, as well as the 32-bit compiler if we install it later)
    • msys (the MSYS environment, and loads of useful non-compiler tools can be installed here, but they shouldn't have any binary dependencies in c:\mingw\64 or c:\mingw\32, or things will get broken!)

MinGW-w64

MinGW-w64 (Minimalist GNU for Windows, 64-bit) is a fork of the earlier MinGW GCC (GNU Compiler Collection: C, C++ and Fortran compilers and associated tools). MinGW-w-64 was carefully extended to include support for 64-bit Windows. Despite the name, MinGW-w64 both runs on and builds binaries for both 32-bit and 64-bit Windows, but you need to make sure you download the right package according to which platform you are running on, and which platform you want to target.

  • Download the TDM-GCC 'MinGW-w64 sjlj' installer for from this page. We used tdm64-gcc-4.7.1-3.exe.
  • Install the TDM-GCC package into the directory c:\mingw\64, selecting to install all components.
  • Open the MSYS shell by double-clicking your new desktop shortcut, and type gcc --version. You should see
john@win7_64 ~
$ gcc --version
gcc.exe (tdm64-1) 4.7.1
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
  • In gedit, open (or create) the file c:\mingw\msys\home\yourusername\.profile (no file extension). In that file, enter the following text, and save the resulting file.
export PATH=$PATH;/c/Python27:/c/Python27/Scripts
alias gedit="/c/Program\ Files\ \(x86\)/gedit/bin/gedit"
alias mingw-get="/c/mingw/32/bin/mingw-get"

TODO: add instructions for also installing the 32-bit compilers, and instructions for getting things working on a 32-bit Windows system.

SCons

We need to custom-build our own SCons installer from the source code, because (last time we checked), the SCons installer didn't correctly detect 64-bit Python on our system.

  • Download, unpack, build and install SCons using the following commands, typed into a new MSYS window.
cd ~
wget http://prdownloads.sourceforge.net/scons/scons-2.3.0.tar.gz
tar zxf scons-2.3.0.tar.gz
python setup.py bdist_wininst
start dist/scons-2.3.0-win-amd64.exe
  • Step through the installer process, then return to your MSYS window and type
cp /c/Python27/Scripts/scons.py /c/Python27/Scripts/scons
  • To ensure that you can do parallel builds (using more than one of your processors at a time), also install pywin32, available from Christophe Gohlke's Python extensions page or from the ASCEND third-party downloads page. Use the amd64 version if you selected 64-bit Python above.
  • Test that SCons is working by typing scons --version. You should get a copyright message including text to show that the you have version 2.3.0 of SCons running:

Dependency Walker

  • Download the 'x64' version of Dependency Walker. We downloaded this file: depends22_x64.zip. If you are also building 32-bit applications, you will also need the i686 version of Dependency Walker.
  • Unpack the .exe and .dll files into the directory c:\mingw\64\bin, for the 64-bit version and/or c:\mingw\32\bin for the 32-bit version. By utting the files here you will automatically get access to the correct version of depends corresponding to the 32- or 64-bit mode you are running via /etc/fstab
  • Test that it's working by typing into MSYS the command depends /mingw/bin/gcc.exe. A window should pop up showing the DLLs that your GCC compiler depends on.

TODO check that two versions of depends.exe are actually required! Perhaps they're not both needed, in which case we could install to c:\mingw\msys\bin.

SWIG

  • Download a pre-compiled 'swigwin' copy of SWIG. We used swigwin-2.0.10.zip. As SWIG is a code pre-processor, it doesn't matter whether it's 32-bit or 64-bit, they will be perfectly equivalent.
  • Create a folder c:\mingw\msys\opt using Windows Explorer, then copy the 'swigwin-2.0.10' folder out of the Zip archive and into the new location c:\mingw\msys\opt\swigwin-2.0.10
  • In MSYS, type gedit ~/.profile, then add the SWIG directory to your PATH in your profile, so that it reads:
export PATH=/c/Python27:/c/Python27/Scripts:$PATH
alias gedit="/c/Program\ Files\ \(x86\)/gedit/bin/gedit"
export PATH=$PATH:/opt/swigwin-2.0.10
  • Close your MSYS window and reopen it, so that the ~/.profile will be re-read.
  • Test that SWIG is accessible by typing swig -version. You should see the version number and some other information output.

Note that SWIG version 2.0.5 was known to cause problems with MinGW-w64.

GTK+2

GTK+ is useful for developing cross-platform graphical user interface (GUI) programs. It works well under Windows, and it is the native GUI toolkit for Ubuntu, Fedora, Mint, and other Linux distributions. It has also been ported to Mac, although getting up and running on that platform is still a bit difficult. The ASCEND project provides some pre-compiled GTK+ installers that make getting started with GTK+ very straightforward. 

export PATH=/c/Python27:/c/Python27/Scripts:$PATH
alias gedit="/c/Program\ Files\ \(x86\)/gedit/bin/gedit"
export PATH=$PATH:/opt/swigwin-2.0.10
export PATH=$PATH:/c/Program\ Files/GTK+-2.22/bin
  • Close your MSYS and re-open it so that the updated ~/.profile is loaded.
  • Test that GTK+ is working by typing either pkg-config gtk+-2.0 --libs or gtk-demo. For the latter, you should see a GUI window pop up that shows off the various GTK+ features.

Note that GTK+3 is now available, however we haven't tested it on Windows up to now.

Subversion

Subversion is a version control software. ASCEND uses it for hosting our source code, as do many other projects, especially on SourceForge.net and Google Code.

  • Download and install the 64-bit version of SlikSVN, Slik-Subversion-1.8.0-x64.msi from the SlikSVN website. You can use the 'typical' settings and default installation location, which is c:\Program Files\SlikSvn
  • Edit your ~/.profile, adding the final line so that you have
export PATH=/c/Python27:/c/Python27/Scripts:$PATH
alias gedit="/c/Program\ Files\ \(x86\)/gedit/bin/gedit"
export PATH=$PATH:/opt/swigwin-2.0.10
export PATH=$PATH:/c/Program\ Files/GTK+-2.22/bin
export PATH=$PATH:/c/Program\ Files/SlikSvn/bin
  • Close your MSYS and re-open it so that the updated ~/.profile is loaded.
  • Check that subversion is working by typing svn --version.

Setup Python for compilation of extensions

Before you can built any Python extensions using MinGW-w64 (which includes via SWIG), some preparation is required. One C header file, pyconfig.h has to be patched, and linker definitions for python27.dll have to be prepared for the compiler. Follow these instructions to sort it out.

First download, build, and install a utility called 'gendef' which builds library *.def files that MinGW-w64 can use when linhing to DLLs. This is because the python27.lib file distributed with Python doesn't work with MinGW-w64. In your MSYS shell, type 

svn co  svn://svn.code.sf.net/p/mingw-w64/code/trunk/mingw-w64-tools/gendef -r5774 ~/gendef
cd ~/gendef 
./configure --prefix=/mingw
make -j4 && make install
gendef --help

Now prepare a new python27.def file from the installed copy of python27.dll in c:\windows\system32:

  • Using Windows Explorer, copy the file c:\windows\system32\python27.dll into the folder c:\Python27\Libs
  • Using MSYS, enter the following:
cd /c/Python27/libs
mv python27.lib old-python27.lib
gendef python27.dll

If this works OK, you should have a new file c:\Python27\Libs\python27.def.

Next, patch the c:\Python27\include\pyconfig.h file:

  • Open the file c:\Python27\include\pyconfig.h using gedit
  • Search for the text #ifdef _WIN64, which in our copy of this file was at line 141, and cut out the following three lines:
#ifdef _WIN64
#define MS_WIN64
#endif
  • Search for the text #ifdef _MSC_VER, which in our copy of this file was at line 107. Paste in the cut-out lines, ABOVE the #ifdef _MSC_VER. Alternatively, if you don't want to do this last step, you could just make sure you define MS_WIN64 before you #include "Python.h" in your source code.

This issue with MS_WIN64 is reported as python bug 4709 in the Python bug tracker, so possibly it will have been fixed if you use a newer version of Python than we tested with.

Usage tips

All done! You may wish to make a backup of your directory c:\mingw, which contains your complete MSYS and MinGW setup, for easy reinstallation later.

When building software on this platform using GNU AutoTools, (ie, the typical './configure && make && make install' instructions), you should specify --prefix=/mingw to ensure files are installed in a path that corresponds to the 32- or 64-bit version of your compiler. This will make library linking much easier, at least while you stay within MSYS. Also, you may need to specify your build environment using the --build flag. For example, 

./configure --prefix=/mingw --build=x86_64-w64-mingw32
make 
make install

This will have the result that libraries and new tools will be installed into c:\mingw\64.

If you ever want to switch building 32-bit code, you simply edit c:\mingw\msys\etc\fstab and change c:/mingw/64 /mingw to c:/mingw/32. Obviously you must install the MinGW-w64 32-bit compiler into that other location in that case. You also need to edit your ~/.profile to point to the alternative Python location, so that you can link against 64-bit Python in any packages you are going to build (also so that SCons scripts correctly determine your 64-bit target architecture). (We're going to try to figure out a way to make these changes simpler, eg by changing only /etc/fstab and not also ~/.profile.)

Example of building a simple program

If everything is working, you should be able to enter the following simple C program into gedit, and save it into c:\mingw\msys\home\yourusername\test.c: 

#include <stdio.h>
int main(){
  printf("Hello, world!\n");
  return 0;
}

Then compile the file and run it by typing 

gcc test.c
./a.exe

And you should see 

Hello, world!

TODO: expand with simple MinGW-w64 example using SCons as the build tool.

Retrieved from "https://ascend4.org/index.php?title=Setting_up_a_MinGW-w64_build_environment&oldid=4363"