Prefix/Cygwin/X

This article describes how to Article description::get Portage to build usable Qt/Gtk-apps on Cygwin.

Preface
Before reading this, you should be familiar with "Gentoo Prefix" and "Prefix on Cygwin".

It would be useful, if you already set up a Cygwin/X system and bootstrapped a Gentoo Prefix on it. However, reading about the theory might be enough, as you'd have to throw it away later anyway. ;-)

Cygwin/X
Before thinking about bootstrapping, a working Cygwin/X system should be at hand. This is not as trivial as one might think. For best results some things should run as daemons, like they do on any GNU/Linux system. But as you are on Windows, this means you have to set up some of the Daemons as Windows Services.

Don't panic, that one is easier than you think.

Basic installation
I am using the cygwin setup.exe directly using a windows command prompt. That is quicker than searching&clicking every package together.

The following commands can be used to install a Cygwin/X system, that already enables Audio, adds a tray icon, dbus support, Kerberos ticketing, private key management, postfix setup, ssh access, system message logging and an xdg-menu-icon.

I bet you didn't know half of that was possible, right? Me neither until a couple of weeks ago...

Important: The *-devel packages are chosen so the later Gentoo prefix does not need concurrent packages to what is running anyway, or what is almost impossible to build on cygwin using portage. We'll tell portage via package.provided what's what.

You can combine the package lists, of course, and install everything at once.

&uArr; Back to top &uArr;


 * Basic systemInstall a rudimentary system that let's you start an X-Server and bootstrap Gentoo:
 * Avahi Support
 * dbus Daemon
 * Tray Indicator
 * Pulse Audio
 * Kerberos
 * Extras

&uArr; Back to top &uArr;

Switching the user context without password
Daemons must be able to switch user context. Either to gain special privileges or to drop unneeded privileges.

For the full details please see the corresponding part in the cygwin ntsec guide.

For the daemons you see above, we need two methods. The first is to configure cyglsa, the second is to add some dummy users to Windows.

&uArr; Back to top &uArr;

Enabling LSA authentication
Start a Cygwin64 Terminal with Administrator privileges and issue the following command:

But before rebooting the machine, you can add the dummy users mentioned above.

&uArr; Back to top &uArr;

Adding dummy users
Actually this is only one group and two users.

If you want to use postfix, do the following:


 * 1) Right click on the Windows start button.
 * 2) Select "Computer Management"
 * 3) Expand "Local Users and Groups"
 * 4) Select "Users"
 * 5) Add a new user named "postfix", any password, with "User cannot change password" and "Password never expires" checked. Make it a member of your general "Users" group.
 * 6) Select "Groups"
 * 7) Add a new group named "postdrop"
 * 8) Add "postfix" and your local user to the group.

At least you need the "nobody" user:


 * 1) Repeat Steps 1-4 above
 * 2) Add a new user named "nobody", any password, with "User cannot change password" and "Password never expires" checked. Make it a member of your general "Users" group.

That's it. The installation of openssh daemon will add a sshd user, and cygserver will add a cyg_server user. Everything else will use the SYSTEM account.

&uArr; Back to top &uArr;

Making the daemons work
In order for the system daemons to work, Windows must be told how to start them, and when. Here are the steps needed for the various daemons. Please make sure that LSA authentication is installed (see above) and Windows was rebooted. Further the dummy users mentioned above should be set up already.

&uArr; Back to top &uArr;

Cygwin Server
This is needed for IPC message queues, semaphores and shared memory support. More information is available on the Cygserver page.


 * 1)  Just follow the on-screen information.
 * 2) Go into the windows services panel and search for "CYGWIN cygserver".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)". The normal automatic start may be too fast. If it is, you get weird UID->Name resolutions.

&uArr; Back to top &uArr;

dbus system bus
It is good to have a system bus. Most applications run fine with a session bus only, but a system bus makes live much easier.


 * 1)  Just follow the on-screen information
 * 2) Go into the windows services panel and search for "CYGWIN D-Bus system service".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)".
 * 4) Open a command (CMD) prompt with administrator privileges and add cygserver to the dependencies: The  is a keyword, there really must be a space between it and the dependency list!

&uArr; Back to top &uArr;

syslog-ng system logger
With this you get back.


 * 1)  Just follow the on-screen information.
 * 2) Go into the windows services panel and search for "CYGWIN syslog-ng".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)".
 * 4) Open a command (CMD) prompt with administrator privileges and add cygserver to the dependencies: The  is a keyword, there really must be a space between it and the dependency list!

&uArr; Back to top &uArr;

Avahi Daemon
Well... yeah... You must decide whether you need this, or not...


 * 1)  Just follow the on-screen information.
 * 2) Go into the windows services panel and search for "CYGWIN Avahi service".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)".
 * 4) Open a command (CMD) prompt with administrator privileges and add dbus to the dependencies: The  is a key... well... you got it, right?

&uArr; Back to top &uArr;

Postfix Mail service
If you want to send mails from the console, or any application that uses postfix, you must install its windows service.


 * 1)  Just follow the on-screen information.
 * 2) Go into the windows services panel and search for "CYGWIN postfix".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)".
 * 4) Open a command (CMD) prompt with administrator privileges and add syslog-ng to the dependencies:

&uArr; Back to top &uArr;

ssh daemon
If you want to use SSH to log into your cygwin, then please, go ahead! :-)


 * 1)  Just follow the on-screen information.
 * 2) Go into the windows services panel and search for "CYGWIN sshd".
 * 3) In its properties Change "Startup type" to "Automatic (Delayed Start)".
 * 4) Open a command (CMD) prompt with administrator privileges and add syslog-ng to the dependencies:

&uArr; Back to top &uArr;

newlib-cygwin - compiling yourself
In the Prefix on Cygwin guide you have read that you need a cygwin1.dll with the "forkables" patches applied.

It should be sufficient to use the pre-compiled one, at least as far as I know. However, if you want to compile your own cygwin1.dll, then this is the section for you.

&uArr; Back to top &uArr;

Needed packages
To be able to successfully configure and compile the project, you will need a few extra packages:

&uArr; Back to top &uArr;

Clone the tree
Which tree you use is up to you and your discretion

These are what the pre-compiled cygwin1.dll mentioned above is made off. These are the same Forkables and Gentoo patches, but rebased on the official master repository, thus including all official fixes.
 * https://github.com/haubi/newlib-cygwin.git (Branch: Gentoo)
 * https://github.com/Yamakuzure/newlib-cygwin.git (Branch: master)

If you need advice, go with the first. I can not guarantee, that my sources do more than compile cleanly, as I do not have the time to check whether all rebased patches are still in whole, and whether the automatic resolving of any conflicts hasn't caused any damage.

&uArr; Back to top &uArr;

Configure and build
As you now have all the packages you need, you can configure and build the whole tree.

Whether you put in these FLAGS or not is up to you, unless you already are in your Gentoo prefix, it should just work without them. But --prefix and --build are important.

When the configure stage is finished, you can build everything. It is best not to push too hard against the windows kernel. The following is well tested on Windows 10 with an 8-core-CPU:

&uArr; Back to top &uArr;

Building from inside the Gentoo prefix
If you build newlib-cygwin from inside you Gentoo prefix, there are two things to take care of:


 * 1) sys-devel/gcc-6.4.0 implicitly sets _FORTIFY_SOURCE to 1 or 2. Both are incompatible with.
 * 2) The default on Gentoo is to build with -fstack-protector, which is incompatible with both  and.

The solution is to configure with:

&uArr; Back to top &uArr;

Install
Yes, you read right. Install this. But not in the root folder, that wouldn't work, because cygwin1.dll is in use. Install it in an image path like this:

The true installation can only happen if absolutely no cygwin process is running. So quit the xdg-menu, close all consoles/windows, and stop all Cygwin services. On Windows 10 you can do the latter using the Task Manager, it has a Services tab. Under Processes you might want to check whether there is no dangling dbus process left behind.

Once everything is stopped, you can use the Windows Explorer or any other file manager to copy the contents of your image directory to your cygwin root directory.

&uArr; Back to top &uArr;

Re-Basing DLLs (fix fork failures)
The dreaded fork failure can happen from time to time. There is much potential for windows software to interfere with Cygwin. Please read "Fixing forkfailures" in the Cygwin FAQ for general information.

However, triggering a full rebase by the Cygwin setup program is meaningless for Gentoo, we are somewhere else. The solution is simple, we do it ourselves.

Find all relevant libraries
There is more to this than just re-basing. There is also the possibility, that a DLL lacks the executable bit. But DLLs must be executable, or it can not be used. Static libraries on the other hand, do not need that bit set.

I have written this little helper script to beat both issues with one bat. Create a list of DLLs to rebase, and fix the executable bit where appropriate. Just adapt the setting to your need.

This will re-create, which lists all possible re-basable libraries.

&uArr; Back to top &uArr;

Do the rebase
The rebase is done in these simple steps:


 * 1) Close all Cygwin processes and log out of your Cygwin X session.
 * 2) End all running cygwin services. The Task Manager in Windows 10 has a "Services" tab you can use.
 * 3) Start a Command Prompt (CMD) with administrator privileges.
 * 4) If the above command tells you that only dash.exe must be running when rebasing, you have some Cygwin processes left. You can kill them (bash.exe, dbus, etc.) using the Task Manager.
 * 1) If the above command tells you that only dash.exe must be running when rebasing, you have some Cygwin processes left. You can kill them (bash.exe, dbus, etc.) using the Task Manager.

You are done. Start all Cygwin Services, you X Session, open a console and continue with your bootstrapping.

Bootstrapping the Prefix
Just do the bootstrap like described on the Project:Prefix/Bootstrap page, unless you do want to start of with some pre-configuration. I prefer the EPREFIX to be "/gentoo", and will refer to it in this way.

If you'd like to use the latest portage tree, you can tell the bootstrap script to do so by issuing: However, I have already had fails due to that download being not what I wanted. It really downloads, which might or might not work. The prefix tree, however, will work.

&uArr; Back to top &uArr;

Things to note down
In case of failure, you can often use the command to finish a merge. But when you are outside the bootstrap script and back on your terminal, the environment is back to normal.

When you have told the script what your EPREFIX should be, it'll print the important environment variables for you. You may want to copy them somewhere safe and use them to prefix a later possible manual run.

These are what I have on my machine: EPREFIX=/gentoo CHOST=x86_64-pc-cygwin PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/Windows MAKEOPTS=-j5

&uArr; Back to top &uArr;

Custom ebuild script for fixing issues
If an ebuild does not finish merging, like gcc always halting at the end of the compile stage, at least on my machine, it may be useful to have a custom ebuild script that takes care of the above environment values.

The following is such a script, that I have in ~/sys/bin, and it works pretty well. At least for me. Adapt and use it if you like.

&uArr; Back to top &uArr;

Possible Preparations
There are a few things you can do before bootstrapping your gentoo prefix. Not much, really, and everything here is purely optional.

&uArr; Back to top &uArr;

Build in RAM
If you have enough RAM, you can have portage do the builds there. Windows does not know tmpfs, but there is a nice freeware tool you can use to setup RAM disks. ImDisk Virtual Disk Driver for Windows can be used to set up a RAM DISK and mount it in /build, or wherever you want. Just do not forget to tell portage about it by adding export PORTAGE_TMPDIR=/build (or wherever you mount your RAM Disk) to your ~/.bashrc. Alternatively you can set up a make.conf to start with, and add the PORTAGE_TMPDIR entry there.

&uArr; Back to top &uArr;

Pre-Setup a make.conf
I'll post here my make.conf for bootstrapping. It adds -march=native to the C[XX]FLAGS, sets CPU_FLAGS_X86 and enables split logs in /gentoo/var/log/portage. Further portage is made cute and rather silent. Maybe you want to remove that part.

If you really want to use this, put it in /gentoo/etc/portage and be damn sure to check CPU_FLAGS_X86 against your CPU!

Oh, and have a look at the localization settings at the bottom! ;-)

&uArr; Back to top &uArr;

Portage tree too old
If any of the problems below signals that your portage tree is too old, either because using didn't do what you expected, or because your  script is too old, you do not have to start over. Just fix this and continue.


 * 1) Download the latest boostrap-prefix.sh script.
 * 2) Restart the script, but kill it with Ctrl+C after it has synced in the latest prefix tree, and starts to continue the bootstrap.
 * 3) If you have binutils-2.28.1 installed, do the following steps:
 * 4) Source  and
 * 5) If you have gcc-5.3.0 installed, do the following steps:
 * 6) Make sure that the following command shows that gcc-6.4.0 is selected:
 * 7) Source  and
 * 1) If you have gcc-5.3.0 installed, do the following steps:
 * 2) Make sure that the following command shows that gcc-6.4.0 is selected:
 * 3) Source  and
 * 1) Source  and

After this you can continue your bootstrap process.

&uArr; Back to top &uArr;

Problems, Breaks, Failures when bootstrapping
There are some things that might fail. Here are the common ones for me.

&uArr; Back to top &uArr;

gcc: Never ending merge
On some rare occasions the merge of sys-devel/gcc will halt after the compilation stage is over.

Luckily this is easy to fix, as long as you noted down the values mentioned above, that were displayed by the bootstrap script. Just substitute the values here with your values.

Or even better: Use my script above, it'll make things easy for you. ;-)

&uArr; Back to top &uArr;

Send Ctrl+C to break the script.
&uArr; Back to top &uArr;

Fix in Stage 2
In Stage 2 use the fixing script above like or issue: $ EPREFIX=/gentoo/tmp \ CHOST=x86_64-pc-cygwin \ PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/windows \ /gentoo/tmp/usr/bin/ebuild /gentoo/usr/portage/sys-devel/gcc/gcc-6.4.0.ebuild merge to finish the merge.

Afterwards you should see: * Switching native-compiler to x86_64-pc-cygwin-6.4.0 ... * Updating LTO plugin symlink in /./gentoo/tmp/usr/x86_64-pc-cygwin/binutils-bin/lib/bfd-plugins    [ ok ]

As this ends stage 2, you can do like portage suggests, before restarting the bootstrap. $ . /gentoo/tmp/etc/profile $ . ~/.bashrc

&uArr; Back to top &uArr;

Fix in Stage 3
If you use the default, or if you changed that but exported the value in your .bashrc, you can proceed like in Stage 2.

Otherwise, if you changed in your make.conf, but did not export the value, you have to fix the build like in Stage 2, but set  to /gentoo/var/tmp like this:

In Stage 3 use the fixing script above like or issue: $ EPREFIX=/gentoo \ CHOST=x86_64-pc-cygwin \ PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/Windows \ PORTAGE_TMPDIR=/gentoo/var/tmp \ /gentoo/tmp/usr/bin/ebuild /gentoo/usr/portage/sys-devel/gcc/gcc-6.4.0.ebuild merge and it will (hopefully) be finished.

This always did the trick for me. This halt happened for me in stage2 and stage3. Well, at least it did not occur in any other situation for me.

After the merge finishes, you should see something like: * Switching native-compiler to x86_64-pc-cygwin-6.4.0 ... [ ok ]

After that you can source and your  and just restart the bootstrap script. It will continue where it left off.

&uArr; Back to top &uArr;

app-arch/xz-utils fail in Stage 3
In stage 2, "" merges just fine, using from binutils-2.29.1 provided by cygwin. But in stage 3 the provided by the available binutils-2.28.1 breaks. It does not recognize the parameter in its call from libtool.

To fix this we are going to force the xz-utils build system to use the cygwin version of.

~ $ EXTRA_ECONF="ac_cv_prog_RC=/usr/bin/windres.exe" ~/sys/bin/do_ebuild /gentoo/usr/portage/app-arch/xz-utils/xz-utils-5.2.3.ebuild clean configure (...) checking for x86_64-pc-cygwin-windres... (cached) /usr/bin/windres.exe (...)

This looks good, so call "" instead of "" to install a working xz-utils. This time it should work.

Afterwards just restart the bootstrap script, it should continue where it left off.

&uArr; Back to top &uArr;

sys-apps/coreutils-8.29 fails in Stage 3
As of writing this, fails with the message: mv: cannot stat '/build/portage/sys-apps/coreutils-8.29/image/gentoo//usr/libexec/coreutils/libstdbuf.dll.a.exe': No such file or directory

Unfortunately the patches and methods to emerge coreutils-8.28 do not work with 8.29 and the only way to get around this, right now, is to use to merge the older version.

Further you need to mask the new coreutils version, at least for now. $ mkdir -p /gentoo/etc/portage/package.mask $ echo ">=sys-apps/coreutils-8.29" >> /gentoo/etc/portage/package.mask/coreutils_limit

child_info_fork::abort: : Loaded to different address
The re-basing did not work as it should have. Before continuing, you should rebase everything.

&uArr; Back to top &uArr;

child_info_fork::abort still happens from time to time!
Yes. I had that, too. This is extremely irritating when the final is running, as it has to be started over and over again.

On my system I found that the antivirus program (Sophos) interfered with the re-basing portage performed. Not always, just from time to time.

Unfortunately I am in a company domain and have no control over the antivirus software.

If you take a look at the Cygwin list of interfering software, our Sophos version is listed, you might find what is sabotaging you here.

The page has a telling paragraph reading:
 * Random fork failures
 * Caused by hook DLLs that load themselves into every process in the system. POSIX fork semantics require that the memory map of the child process must be an exact duplicate of the parent process' layout. If one of these DLLs loads itself at a different base address in the child's memory space as compared to the address it was loaded at in the parent, it can end up taking the space that belonged to a different DLL in the parent. When Cygwin can't load the original DLL at that same address in the child, the fork call has to fail.
 * Caused by hook DLLs that load themselves into every process in the system. POSIX fork semantics require that the memory map of the child process must be an exact duplicate of the parent process' layout. If one of these DLLs loads itself at a different base address in the child's memory space as compared to the address it was loaded at in the parent, it can end up taking the space that belonged to a different DLL in the parent. When Cygwin can't load the original DLL at that same address in the child, the fork call has to fail.

However, if you, like me, can not turn off the offending software, here is what to do:


 * 1) Complete the failed build using  provided above.
 * 2) Continue the merge with:
 * 3) The final package to merge will most probably be gcc, which might or might not hang. See above
 * 4) That's it, your system rebuild is finished. The final steps the script would have done, which you have to do now manually, are:

Congratulations, your bootstrap is finished!

&uArr; Back to top &uArr;

After the bootstrap
Once your bootstrapping is finished, it is time to update your configuration a bit and install some extra tools. The first tool I always start with, is ; it makes life so much easier...

&uArr; Back to top &uArr;

Update make.conf
Apart from what you might want to add, here are some information about what I add to it.


 * 1) USE flags
 * I tend to always set almost all flags I want at once. The only flags I do not add right now are, $ and , as these would pull in a myriad of packages. Too much right now.
 * So basically I substitute my basic USE flag settings with:
 * 1) Hardware, provided by cygwin
 * If you ask what video card cygwin uses, it is:
 * 1) Python targets
 * Python 3.x is too much pain to upgrade to, so I explicitly limit it to 2.7:

&uArr; Back to top &uArr;

Generate your repos.conf
If you must or want to use webrsync, maybe because your companies firewall blocks rsync, you have to create your own repos.conf:

&uArr; Back to top &uArr;

Install app-portage/eix
As I said, eix makes your life much easier. But we need to prepare some things first.

As you might remember, we installed icu-devel right in the beginning. To use it, add an entry to like this: We use Cygwins icu-devel package here, because building icu ourselves is a difficult thing to do, and packages using it will most likely fail to successfully emerge.

For eix to be installable, you have to accept some keywords for it: Now you should be ably to simply, followed by a to build your first database.

If you stumbled over, don't worry. It just installs three shell scripts. Two are in /gentoo/etc/init.d and ignored (not usable anyway) in a prefix. The third is and eix uses it to safely create.

&uArr; Back to top &uArr;

Add eix-update to postsync.d
Unfortunately it is not a good idea to use in a cygwin prefix. It just doesn't work as expected, so we simply call and  after every : Now, whenever you have synced, but you should not do this yet, eix will update itself.

&uArr; Back to top &uArr;

Fix eclasses after each sync
You may find this curious, but this page is about getting your Gentoo prefix to let you emerge Qt apps.


 * To use Qt, you need cmake. And unfortunately the cmake.eclass is broken for Cygwin.
 * To actually build Qt, you need an awful lot of patches and a fix to the qt5-build.eclass as well.
 * And if you plan to build any KDE apps, a fix to kde5-functions.eclass is also needed.

Luckily I will show you a way how this happens automatically after each, so you do not have to do this by hand each time.

&uArr; Back to top &uArr;

Prepare the postsync.d script
After each is completed, the scripts in the  folder are executed. We already use this for eix, let's use this for our eclass fixes as well.

&uArr; Back to top &uArr;

Fix cmake-utils.eclass for Cygwin
CMake on Cygwin prefix does not understand CMAKE_PLATFORM_REQUIRED_RUNTIME_PATH, so it does not add the supplied paths as intended, but simply dumps them to the command line, which leads to the configure stage to fail.

The following patch fixes that: fix_cmake-utils.eclass_for_cygwin.patch Just place the file into your prefix path.

&uArr; Back to top &uArr;

Fix qt5-build.eclass for Cygwin
For Qt5 there is more to do, because we have two problems:


 * 1) The qt5-build.eclass imposes strict but sane defaults, and makes the Qt5 build system to heed the *FLAGS we set in our make.conf.
 * 2) The Qt5 build systems Cygwin support is utterly broken and must be patched.

Unfortunately the latter is futile, thanks to the first. This is because all the settings needed for Cygwin are ignored, thanks to the qt5-build.eclass.

So first, place fix_qt5-build.eclass_for_cygwin.patch into your prefix directory.

Second, we have to place the patches:

And finally, a few Qt5 packages need some extra patching:

&uArr; Back to top &uArr;

Fix kde5-functions.eclass for Cygwin
In some cases a build can break, because there is a tiny little oversight in kde5-functions.eclass. Just place fix_kde5-functions.eclass_for_cygwin.patch in your prefix directory and the problem is fixed with each sync.

&uArr; Back to top &uArr;

Sync your tree
Yes. Now is the time to sync your tree. And after you did an, you should have seen eix updating, and the process finishing with:

&uArr; Back to top &uArr;

Preparations before your first @world update
Before going on and updating your, you just synced your tree, you have to do some preparations.

First, you might want to install $, which will provide the formidable tool.

Then we have to tell Portage, that it should use some of the packages Cygwin provides. Generally speaking there are three reasons to do so:
 * 1) Packages like, that need running Windows services, have to be provided by Cygwin.
 * 2) A few Packages make no sense or even can not be installed, like  or.
 * 3) Some packages are just not easy to emerge, or provoke future emerge failurs, like.

The resulting file looks like this: (At the moment. Likely to change!)

As you can see this lists some packages we do not have installed yet using Cygwin. So here we go:

Before your can really do your first @world update, two more packages need to be un-keyworded:
 * If you have USE="kerberos"
 * If you have USE="git"

If you have done everything like I have recorded above, ~76 packages should now be listed by your update. A lot of these will be from USE flag changes, so it is good to get things straight now, before we carry on.

&uArr; Back to top &uArr;

fork failures when updating packages (fork: Resource temporarily unavailable)
Like in bootstrapping, this can happen again, if you have interfering software installed.

Something hooked into a dll that was just installed by portage (like cygncurses6.dll) and sabotaged the rebase.


 * 1) Try to finish the merge by calling
 * 2) If this worked, fine. If not, you have to do a full rebase.

&uArr; Back to top &uArr;