Mingw

From Gentoo Wiki
Jump to: navigation, search

MinGW (historically, MinGW32) is a way to cross-compile Windows binaries on Linux or any other OS. It can also be used natively on Windows to avoid using Visual Studio, etc.

Installation

Prerequisites

To install the MinGW32 Toolchain, start with emerging the crossdev tool:

root #emerge --ask sys-devel/crossdev

Emerge

Crossdev will automatically create /etc/portage/package.keywords/cross-i686-w64-mingw32 and /etc/portage/package.use/cross-i686-w64-mingw32. Since by default some critical use flags like sanitize, fortran or vtv are not disabled it might be necessary to override the auto created use flags by

root #echo "cross-i686-w64-mingw32/gcc -fortran -vtv -sanitize" >> /etc/portage/package.use/cross-i686-w64-mingw32-gcc

Now with the crossdev tool installed, emerge the mingw32 toolchain:

root #crossdev --ov-output /usr/portage i686-w64-mingw32
Note
This article assumes a 32-bit toolchain will be built. To compile for a 64-bit target instead, replace the crossdev i686-w64-mingw32 target in the command above with x86_64-w64-mingw32.
Note
The CHOST recommendation has recently changed in this document as a number of packages (ie dev-libs/glib) require the newer API available in mingw64-runtime. If you wish to use the old mingw-runtime for 32-bit, use the target CHOST value of i686-pc-mingw32.

Adding the --ex-insight or --ex-gcc options may cause issues; they have been known to not build. --ex-gdb will enable GDB and likely will work, but it is not very useful on Linux because MinGW GCC by default makes PE's (EXE files), not ELF files, and gdb has no idea how to run an EXE file on Linux. A remote debugger (with a Windows target machine) is a possibility but a long shot.

Notes about the toolchain:

  • GCJ sources will not compile due to missing makespec files that do not get installed (copying from MinGW from Windows does not work either).
  • Sanitize may cause compilation failures for at least gcc-4.9, maybe newer.
  • OpenMP was once forcefully disabled in the ebuild, but now honors the use flag setting. However it may still cause compilation issues when set.

libssp

The GCC USE flag sys-devel/gcc[libssp] has been masked, since it is usually provided by libc. Apparently msvcrt does not provide libssp, so it is recommended to re-enable this USE flag for cross compilation (see package.use.mask):

FILE /etc/portage/profile/package.use.mask/mingw-libssp
cross-i686-w64-mingw32/gcc -libssp
cross-x86_64-w64-mingw32/gcc -libssp
FILE /etc/portage/profile/package.use.force/mingw-libssp
cross-i686-w64-mingw32/gcc libssp
cross-x86_64-w64-mingw32/gcc libssp

Usage

Portage

Some things work. Most things do not. Try with USE="-*" after a failed build, then selectively add USE flags you need. If that does not work, then you probably cannot use Portage to install the package desired for use with MinGW.

make.conf and profile changes

Note
If crossdev didn't create /usr/i686-w64-mingw32/etc/portage/profile/ directory then the emerge-wrapper command may be needed to initialize it:
root #emerge-wrapper --target i686-w64-mingw32 --init

Various ebuilds support mingw or win32 targets, but different build systems often need different indicators. Ensuring the following are set in /usr/i686-w64-mingw32/etc/portage/profile/make.conf should allow most build systems to detect the proper target. Note, some of these may have already been set by crossdev:

FILE /usr/i686-w64-mingw32/etc/portage/profile/make.confProfile-level variables
ARCH=x86
ELIBC="mingw"
KERNEL="Winnt"
Note
As of spring 2016, the default embedded profile that /usr/i686-w64-mingw32/etc/portage/profile/make.profile points to actually excludes all three of the above variables from being set in make.conf. In order to rectify this, at this point in time the easiest way is to use the prefix/windows/winnt profile
root #rm /usr/i686-w64-mingw32/etc/portage/make.profile && ln -s /usr/portage/profiles/prefix/windows/winnt/ /usr/i686-w64-mingw32/etc/portage/make.profile

and set the ARCH variable via /usr/i686-w64-mingw32/etc/portage/profile/make.defaults

FILE /usr/i686-w64-mingw32/etc/portage/profile/make.defaults
ARCH=x86
Another possibility is to delete the make.profile symlink and copy /usr/portage/profiles/embedded directly in its place. Then you can edit make.defaults to remove the above three variables from the PROFILE_ONLY_VARIABLES list at the bottom of the file.

Mingw64-runtime and the cross-toolchain do not provide any libgcc_s_*.dll files, and without an external source for these files (eww) there will be issues trying to execute what is built by the cross-toolchain. Fortunately, there's a workaround in the form of LDFLAGS -static-libgcc and -static-libstdc++, however due to the fact that these non-standard flags tend to get stripped out of builds, you need to perform some trickery. Add the following to the make.conf:

FILE /usr/i686-w64-mingw32/etc/portage/profile/make.conf
# Workaround missing libgcc_s_*.dll files by statically linking libc and libstdc++
CC="${CHOST}-gcc -static-libgcc"
CXX="${CHOST}-g++ -static-libgcc -static-libstdc++"

USE flags can be set globally in make.conf or per-package in package.use; as the builds are for win32 it likely makes sense to globally disable some flags, such as USE="-X" and globally enable USE="win32" in case any packages support it.

Finally, you likely want to make sure that the code you compile will actually run on the targets you plan to execute it on. This means setting appropriate -march and -mtune values in the CFLAGS variable for the target platform:

FILE /usr/i686-w64-mingw32/etc/portage/profile/make.conf
CFLAGS="-march=i586 -mtune=generic -O2 -pipe"
CXXFLAGS="${CFLAGS}"

Emerging packages

Cross emerging is done by i686-w64-mingw32-emerge. For example, to emerge the sys-libs/zlib package, use:

root #i686-w64-mingw32-emerge sys-libs/zlib

Alternatively:

root #emerge-wrapper --target i686-w64-mingw32 -av sys-libs/zlib
Note
You can ged rid of emerge's news message by deleting the content of /usr/i686-w64-mingw32/var/lib/gentoo/news/news-gentoo
root #echo "" > /usr/i686-w64-mingw32/var/lib/gentoo/news/news-gentoo.unread

Using Portage, you may run into problems such as the following:

  • Application wants GDBM (see below).
  • Application wants to link with ALSA/OSS/Mesa/other library only useful to X or Linux.
  • Ebuild of application doesn't contain the necessary configuration option to support a mingw or win32 target.
  • Application is an unnecessary utility script, such as gentoo-functions or eselect.
  • An ebuild inherits multilib and specifies MULTILIB_CHOST_TOOLS.

In the multilib case, emerge wants to move the executables specified in MULTILIB_CHOST_TOOLS. But when cross compiling with mingw32 the executables receive an extension .exe and emerge cannot find the file without extension and fails.

At this point the only fix I know is overlaying (see below) a custom ebuild, appending the extension .exe to all files in MULTILIB_CHOST_TOOLS. This concerns for example dev-libs/glib, media-libs/fontconfig, x11-libs/gdk-pixbuf, x11-libs/cairo/cairo and x11-libs/gtk+.

The main techniques to tweak ebuilds to make them work are

Overriding use flags, keywords and configuration options

To override use flags and keywords, simply use /etc/portage/package.use/ and /etc/portage/package.keywords/ respectively. For the configuration options, we can tell emerge to use a package specific file defining environment variables (see package.env). For example, if we want to configure x11-libs/cairo with --with-target=win32, we create

FILE /usr/i686-w64-mingw32/etc/portage/env/cairo.confEnvironment overrides for configure options
EXTRA_ECONF="--with-target=win32"
FILE /usr/i686-w64-mingw32/etc/portage/package.env/cairoSpecifying cairo.conf as package specific file defining overrides for cairo
x11-libs/cairo cairo.conf
Overlaying

If issues cannot simply be fixed by overriding configure options, in some cases we have to override ebuilds. In order to do that we create a custom local overlay. Since /usr/i686-w64-mingw32/usr/portage is empty, we will use this path. Create the following files

FILE /usr/i686-w64-mingw32/usr/portage/metadata/layout.conf
masters = gentoo
FILE /usr/i686-w64-mingw32/usr/portage/profiles/repo_name
cross-i686-w64-mingw32
FILE /usr/i686-w64-mingw32/etc/portage/repos.conf
[DEFAULT]
main-repo = gentoo

[cross-i686-w64-mingw32]
location = /usr/i686-w64-mingw32/usr/portage/
masters = gentoo
auto-sync = no

Portage will then use our custom ebuilds in the /usr/i686-w64-mingw32/usr/portage/ folder.

Notes on specific packages

app-admin/eselect

This package brings in a number of system dependencies that are just plain not needed to build win32 software, and at the time of writing many of them (like python) fail to emerge. However, as the binary is called during phase functions of other ebuilds you do want, a simple package.provided entry does not suffice to get rid of it. Instead, I recommend overlaying your own app-admin/eselect ebuild that installs a dummy eselect binary, something that will do nothing yet always return success. This is a dirty hack that certainly has drawbacks, but it at least allows the meat of slotted packages to be emerged.

The ebuild could look for example like this

FILE /usr/i686-w64-mingw32/usr/portage/app-admin/eselect/eselect-1.4.6.ebuild
EAPI=5

DESCRIPTION="Dummy binary"

SLOT="0"
KEYWORDS="~alpha ~amd64 ~arm ~arm64 ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~ppc-aix ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~x64-freebsd ~x86-freebsd ~hppa-hpux ~ia64-hpux ~x86-interix ~amd64-linux ~arm-linux ~ia64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~m68k-mint ~sparc-solaris ~sparc64-solaris ~x64-solaris ~x86-solaris"

src_unpack() {
	mkdir ${WORKDIR}/${P}
}

src_install() {
	mkdir -p ${D}/usr/bin/
	echo ":" > ${D}/usr/bin/${PN}
	chmod +x ${D}/usr/bin/${PN}
}
sys-apps/gentoo-functions

This is another one of those necessary tool dependencies that isn't really needed in a mingw cross-build environment. Although mostly implemented in shell, there is a single compiled binary that fails due to missing POSIX API stuff, /sbin/consoletype. This package may be something that can be package.provided away, but to be on the safe side one can also overlay this ebuild and install a dummy script that echo's 'serial' and exists with code 1, in place of compiling consoletype.

dev-util/gtk-update-icon-cache

gtk-update-icon-cache is a tool that various packages inheriting the gnome eclasses will call in their pkg_postinst phase functions. Although it may be a good idea to install it for use within the win32 target environment, the resulting binary cannot be run in phase functions and so failures will often occur. Another dummy-script-installing overlay package can get around this issue.

OpenSSL

Follow this guide: [1]

sys-libs/ncurses

Ncurses is a very finicky package, mostly due to the fact that it's build system was generated using a custom-forked version of autotools. At this time of writing, sys-libs/ncurses-5.9-r5 is stable and a static-only installation will emerge with EXTRA_ECONF="--enable-term-driver --enable-sp-funcs --without-shared" and USE="static-libs", but ncurses-6.0 will not compile.

sys-libs/readline

sys-libs/readline is another finicky package, in part because it depends on ncurses. Only ~sys-libs/readline-6.2_p5 will build successfully, newer versions need a lot of patching. Further, due to ncurses being limited to a static-only installation, readline must also be built static-only using EXTRA_ECONF="--disable-shared" and USE="static-libs".

x11-libs/cairo

Cairo is well supported but the ebuilds currently do not provide a USE flag for the win32 target. Specifying EXTRA_ECONF="--with-target=win32" and ensuring USE="-X -aqua -xcb -x11-xcb" will address this for now.

If the plan is to emerge x11-libs/gtk+, then we abuse the aqua use flag (both packages do not provide a win32 use flag) in order to avoid forced X11 dependencies ans set USE="aqua" for both packages. This will enable quartz support via configure options which we have to suppress by specifying EXTRA_ECONF="--enable-quartz=no --enable-quartz-image=no".

Note
Support for a win32 use flag may soon be added to cairo in the main Gentoo repository.
x11-libs/gdk-pixbuf

This package builds as-is without any modification, however there are two minor issues related to using the package:

  • The pkg_postinst phase is unable to run 'gdk-pixbuf-query-loaders' to generate the loaders.cache file, which means that this will need to be done by hand using wine, via something like
root #wine /usr/i686-w64-mingw32/usr/bin/gdk-pixbuf-query-loaders.exe >/usr/i686-w64-mingw32/usr/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
  • The paths used by gdk-pixbuf at runtime to find the various loader DLLs is absolute, meaning that they will need to be installed on target win32 systems at [drive]:\usr\lib\gdk-pixbuf-2.0\2.10.0\loaders.

is possible to circumvent both of these issues by building the gdk-pixbuf with EXTRA_ECONF="--with-included-loaders=yes", as this will include the loader code directly in the main gdk-pixbuf dll.

x11-libs/gtk+

As touched on in the section about cairo above, in order to avoid a lot of X11 deps, gtk+ needs to be built with USE="aqua" and EXTRA_ECONF="--with-gdktarget=win32" for gtk+:2 or EXTRA_ECONF="--enable-win32-backend --disable-quartz-backend" for gtk+:3.

If build failures related to missing symbols are seen in the libraries at installation time, this may be related to a need to clear the gtk.def file so that it can be regenerated properly by the build system. An easy way to do this without overlaying the ebuilds is to use the following script snippet in /usr/i686-w64-mingw32/etc/portage/bashrc:

FILE /usr/i686-w64-mingw32/etc/portage/bashrc
case "${CATEGORY}/${PN}" in
x11-libs/gtk+)
        case "${EBUILD_PHASE}" in
        prepare)
                einfo "Cleaning out gtk/gtk.def to aid in mingw out-of-source builds"
                rm -f "${S}"/gtk/gtk.def
                ;;
        esac
        ;;
esac
Note
x11-libs/gtk+-3.18 and newer needs the function cairo_win32_surface_create_with_format, which reportedly will not be available until x11-libs/cairo-1.15.2 or newer.
Note
There are currently collisions between gtk+:2 and gtk+:3 on the file ${EROOT}/usr/bin/gailutil.def; this needs to be resolved by hand if both slots are desired at the same time.
Note
Support for a win32 use flag may soon be added to cairo in the gentoo repo.
GDBM

These are "Standard GNU database libraries" according to Portage. Many libraries and applications depend on this. The package reportedly compiled successfully compiled in the past, but the current versions in Portage do not compile due to the package requiring a POSIX environment (which mingw is not). Patching is very much needed.

FILE build.logexcerpt
i686-w64-mingw32-gcc -c -I. -I. -march=k8 -msse3 -O2 -pipe gdbmfetch.c  -DDLL_EXPORT -DPIC -o .libs/gdbmfetch.lo                                                                                                     
gdbmopen.c: In function 'gdbm_open':                                                                      
gdbmopen.c:171: error: storage size of 'flock' isn't known                                                
gdbmopen.c:171: error: 'F_RDLCK' undeclared (first use in this function)                                  
gdbmopen.c:171: error: (Each undeclared identifier is reported only once                                  
gdbmopen.c:171: error: for each function it appears in.)                                                  
gdbmopen.c:171: error: 'F_SETLK' undeclared (first use in this function)                                  
gdbmopen.c:177: error: storage size of 'flock' isn't known                                                
gdbmopen.c:177: error: 'F_WRLCK' undeclared (first use in this function)

To get around this problem for the moment, try building with USE="-*".

SDL tutorial example

Emerge SDL:

root #i686-w64-mingw32-emerge media-libs/libsdl

Try compiling this source code (save to test.c).

FILE test.c
#include <SDL/SDL.h>
#include <windows.h>

void cool_wrapper(SDL_Surface **s, int flags) {
  *s = SDL_SetVideoMode(640, 480, 32, flags);
  return;
}

int main(int argc, char *argv[]) {
  int flags;
  SDL_Surface *s;

  SDL_Init(SDL_INIT_VIDEO);

  flags  = SDL_OPENGL;            /* Enable OpenGL */
  flags |= SDL_GL_DOUBLEBUFFER;  /* Enable double-buffering */
  flags |= SDL_HWPALETTE;        /* Enable storing palettes in hardware */
  flags |= SDL_RESIZABLE;        /* Enable window resizing */         
  
  cool_wrapper(&s, flags);
  Sleep(5000);

  SDL_FreeSurface(s);

  SDL_Quit();

  return 0;
}

Use the following command to build:

user $i686-w64-mingw32-gcc -o test.exe test.c `/usr/i686-w64-mingw32/usr/bin/sdl-config --libs`

Test with Wine (requires SDL.dll to be somewhere in Wine's %PATH%, which includes the same directory as the EXE):

user $cp /usr/i686-w64-mingw32/usr/bin/SDL.dll .
user $wine test.exe

If you get a window named SDL_app, then it worked. The window will automatically exit after about 5 seconds (the Windows Sleep() function halts execution for 5000 milliseconds).

Porting POSIX threads to Windows

Windows thread functions seem to work fine with MinGW. The following example code will compile without error:

FILE win32_threads.c
#include <windows.h>                                        
#include <stdio.h>                                          
#include <stdlib.h>                                         

#define NUM_THREADS 5

DWORD print_hello(LPVOID lpdwThreadParam);

int main(int argc, char *argv[]) {
  int i;
  DWORD dw_thread_id;

  for (i = 0; i < NUM_THREADS; i++) {
    if (CreateThread(NULL,                                 /* Default security level */
                     0,                                    /* Default stack size */
                     (LPTHREAD_START_ROUTINE)&print_hello, /* Routine to execute */
                     (LPVOID)&i,                           /* Thread paramater */
                     0,                                    /* Run immediately */
                     &dw_thread_id                         /* Thread ID */
                     ) != NULL) {
      printf("In main: Creating thread %d\n", i);
      Sleep(1000);
    }
    else {
      printf("Error: Failed to create the %d\n", i);
      exit(EXIT_FAILURE);
    }
  }

  exit(EXIT_SUCCESS);
}

/* Thread routine */
DWORD print_hello(LPVOID lpdwThreadParam) {
  printf("Thread #%d responding\n", *(int*)lpdwThreadParam);
  return 0;
}

Compile with:

user $i686-w64-mingw32-gcc -o win32_threads.exe win32_threads.c

(The call to Sleep() will make the thread creation a little more closer to POSIX, more in order, and there will not be duplicate runs.)

However, many applications rely upon POSIX threads and do not have code for Windows thread functionality. The POSIX Threads for Win32 project provides a library for using POSIX thread-like features on Windows (rather than relying upon Cygwin). It basically wraps POSIX thread functions to Win32 threading functions (pthread_create()->CreateThread() for example). Be aware that not everything is implemented on either end (however do note that Chrome uses this library for threading on Windows). Regardless, many ported applications to Windows end up using POSIX Threads for Win32 because of convenience. With this library you can get the best of both worlds as Windows thread functions work fine as show above.

To get Pthreads for Win32:

  1. Go to the Sourceware FTP and download the header files to your includes directory for MinGW (for me this is /usr/i686-w64-mingw32/usr/include).
  2. Go to the Sourceware FTP and download only the .a files to your lib directory for MinGW (for me this is /usr/i686-w64-mingw32/usr/lib).'
  3. At the same directory, get the DLL files (only pthreadGC2.dll and pthreadGCE2.dll; others are for Visual Studio) and place them in the bin directory of your MinGW root (for me this is /usr/i686-w64-mingw32/usr/bin).

Example POSIX threads code:

FILE win32_posix_threads.c
#include <pthread.h>                                         
#include <stdio.h>                                           
#include <stdlib.h>                                          

#define NUM_THREADS 5

void *print_hello(void *thread_id) {
  long tid;                         
  tid = (long)thread_id;            
  printf("Thread #%ld responding.\n", tid);
  pthread_exit(NULL);                      
  return NULL;                             
}                                          

int main(int argc, char *argv[]) {
  pthread_t threads[NUM_THREADS];
  pthread_attr_t attr;
  int rc, status;
  long i;

  for (i = 0; i < NUM_THREADS; i++) {
    printf("In main: creating thread %ld\n", i);
    rc = pthread_create(&threads[i], NULL, print_hello, (void *)i);
    if (rc) {
      printf("Error: return code from pthread_create() is %d\n", rc);
      exit(EXIT_FAILURE);
    }
  }

  pthread_attr_destroy(&attr);
  for (i = 0; i < NUM_THREADS; i++) {
    rc = pthread_join(threads[i], (void **)&status);
    if (rc) {
      printf("Error: return code from pthread_join() is %d\n", rc);
      exit(EXIT_FAILURE);
    }
    printf("Completed join with thread %d, status = %d\n", i, status);
  }
  pthread_exit(NULL);

  exit(EXIT_SUCCESS);
}

Compile with:

user $i686-w64-mingw32-gcc -o posix_threads.exe -mthreads posix_threads.c -lpthreadGC2
Note
It is VERY important that -lpthreadGC2 or -lpthreadGCE2 is at the END of the command.

With i686-w64-mingw32-objdump -p posix_threads.exe we can see that we need pthreadGC2.dll. If you linked with -lpthreadGCE2 (exception handling POSIX threads), you will need mingwm10.dll, pthreadGCE2.dll, and possibly libgcc_s_sjlj-1.dll (last one only if you do not compile with CFLAG -static-libgcc with g++).

Copy the DLL file(s) required to the directory and test with Wine. For example:

user $cp /usr/i686-w64-mingw32/usr/bin/pthreadGC2.dll .
user $wine posix_threads.exe

If all goes well, you should see output similar to the following:

In main: creating thread 0                                                                                         
In main: creating thread 1                                                                                         
Thread #0 responding.                                                                                              
In main: creating thread 2                                                                                         
Thread #1 responding.                                                                                              
In main: creating thread 3                                                                                         
Thread #2 responding.                                                                                              
In main: creating thread 4                                                                                         
Thread #3 responding.                                                                                              
Thread #4 responding.                                                                                              
Completed join with thread 0, status = 0                                                                           
Completed join with thread 1, status = 0                                                                           
Completed join with thread 2, status = 0                                                                           
Completed join with thread 3, status = 0                                                                           
Completed join with thread 4, status = 0   
Note
You will probably always want to include -mthreads as a CFLAGS value for any code that relies on thread-safe exception handling. From the man page:
  • -mthreads - Support thread-safe exception handling on MinGW 32. Code that relies on thread-safe exception handling must compile and link all code with the -mthreads option. When compiling, -mthreads defines:
  • -D_MT; when linking, it links in a special thread helper library
  • -lmingwthrd which cleans up per thread exception handling data.

Wine and %PATH%

Like Windows, Wine supports environment variables. You may specify the path of your DLLs (for example, the MinGW bin directory) in the registry at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment (for me this value would be Z:\usr\i686-w64-mingw32\usr\bin). I recommend against this as you might forget to distribute DLLs with your application binaries.

No need for -lm

If you #include <math.h> and make use of any of its functions, there is no need to link with the standard C math library using the -lm switch with gcc or .

DirectX

DirectX 9 headers and libs are included. Link with -ldx9. For the math functions (such as MatrixMult, unlike Windows, you need to dynamically link with -ld3dx9d and then include d3dx9d.dll (where you get this file SHOULD be from Microsoft's SDK). This is the same for DirectX 8.

There is no support for DirectX 10 or 11 yet. Minimal support for Direct2D has been implemented via a patch (search the official mailing list of MinGW).

Removal

root #crossdev -C i686-w64-mingw32

If files are left over (such as libraries and things that have been added), a prompt will occur to remove the /usr/i686-w64-mingw32 directory recursively.

Troubleshooting

Emerging a toolchain failed with error: Missing digest for *.ebuild

Add the following to the crossdev overlay metadata:

FILE layout.conf
thin-manifests = true