Prefix/Cygwin/X

This article describes how to Article description::get Portage to build usable Qt/Gtk-apps on Cygwin.

Preface
Before reading this, you should be familiar with "Gentoo Prefix" and "Prefix on Cygwin".

It would be useful, if you already set up a Cygwin/X system and bootstrapped a Gentoo Prefix on it. However, reading about the theory might be enough, as you'd have to throw it away later anyway. ;-)

Cygwin/X
Before thinking about bootstrapping, a working Cygwin/X system should be at hand. This is not as trivial as one might think. For best results some things should run as daemons, like they do on any GNU/Linux system. But as you are on Windows, this means you have to set up some of the Daemons as Windows Services.

Don't panic, that one is easier than you think.

Basic installation
I am using the cygwin setup.exe directly using a windows command prompt. That is quicker than searching&clicking every package together.

The following commands can be used to install a Cygwin/X system, that already enables Audio, adds a tray icon, dbus support, Kerberos ticketing, private key management, postfix setup, ssh access, system message logging and an xdg-menu-icon.

I bet you didn't know half of that was possible, right? Me neither until a couple of weeks ago...

Important: The *-devel packages are chosen so the later Gentoo prefix does not need concurrent packages to what is running anyway, or what is almost impossible to build on cygwin using portage. We'll tell portage via package.provided what's what.

You can combine the package lists, of course, and install everything at once.

Basic system
Install a rudimentary system that let's you start an X-Server and bootstrap Gentoo:

Switching the user context without password
Daemons must be able to switch user context. Either to gain special privileges or to drop unneeded privileges.

For the full details please see the corresponding part in the cygwin ntsec guide.

For the daemons you see above, we need two methods. The first is to configure cyglsa, the second is to add some dummy users to Windows.

Enabling LSA authentication
Start a Cygwin64 Terminal with Administrator privileges and issue the following command:

But before rebooting the machine, you can add the dummy users mentioned above.

Adding dummy users
Actually this is only one group and two users.

If you want to use postfix, do the following:


 * 1) Right click on the Windows start button.
 * 2) Select "Computer Management"
 * 3) Expand "Local Users and Groups"
 * 4) Select "Users"
 * 5) Add a new user named "postfix", any password, with "User cannot change password" and "Password never expires" checked. Make it a member of your general "Users" group.
 * 6) Select "Groups"
 * 7) Add a new group named "postdrop"
 * 8) Add "postfix" and your local user to the group.

At least you need the "nobody" user:


 * 1) Repeat Steps 1-4 above
 * 2) Add a new user named "nobody", any password, with "User cannot change password" and "Password never expires" checked. Make it a member of your general "Users" group.

That's it. The installation of openssh daemon will add a sshd user, and cygserver will add a cyg_server user. Everything else will use the SYSTEM account.

newlib-cygwin - compiling yourself
In the Prefix on Cygwin guide you have read that you need a cygwin1.dll with the "forkables" patches applied.

It should be sufficient to use the pre-compiled one, at least as far as I know. However, if you want to compile your own cygwin1.dll, the this is the section for you.

Needed packages
To be able to successfully configure and compile the project, you will need a few extra packages:

Clone the tree
Which tree you use is up to you and your discretion

These are what the pre-compiled cygwin1.dll mentioned above is made off. These are the same Forkables and Gentoo patches, but rebased on the official master repository, thus including all official fixes.
 * https://github.com/haubi/newlib-cygwin.git (Branch: Gentoo)
 * https://github.com/Yamakuzure/newlib-cygwin.git (Branch: master)

If you need advice, go with the first. I can not guarantee, that my sources do more than compile cleanly, as I do not have the time to check whether all rebased patches are still in whole, and whether the automatic resolving of any conflicts hasn't caused any damage.

Configure and build
As you now have all the packages you need, you can configure and build the whole tree.

Whether you put in these FLAGS or not is up to you. it should just work without them. But --prefix and --build are important.

When the configure stage is finished, you can build everything. It is best not to push too hard against the windows kernel. The following is well tested on Windows 10 with an 8-core-CPU:

Install
Yes, you read right. Install this. But not in the root folder, that wouldn't work, because cygwin1.dll is in use. Install it in an image path like this:

The true installation can only happen if absolutely no cygwin process is running. So quit the xdg-menu, close all consoles/windows, and stop all Cygwin services. On Windows 10 you can do the latter using the Task Manager, it has a Services tab. Under Processes you might want to check whether there is no dangling dbus process left behind.

Once everything is stopped, you can use the Windows Explorer or any other file manager to copy the contents of your image directory to your cygwin root directory.

Bootstrapping the Prefix
Just do the bootstrap like described on the Project:Prefix/Bootstrap page, unless you do want to start of with some pre-configuration. I prefer the EPREFIX to be "/gentoo", and will refer to it in this way.

If you'd like to use the latest prefix tree, you can tell the bootstrap script to do so by issuing:

Thinks to note down
In case of failure, you can often use the command to finish a merge. But when you are outside the bootstrap script and back on your terminal, the environment is back to normal.

When you have told the script what your EPREFIX should be, it'll print the important environment variables for you. You may want to copy them somewhere safe and use them to prefix a later possible manual run.

These are what I have on my machine: EPREFIX=/gentoo CHOST=x86_64-pc-cygwin PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/Windows MAKEOPTS=-j5

Custom ebuild script for fixing issues
If an ebuild does not finish merging, like gcc always halting at the end of the compile stage, at least on my machine, it may be useful to have a custom ebuild script that takes care of the above environment values.

The following is such a script, that I have in ~/sys/bin, and it works pretty well. At least for me. Adapt and use it if you like.

Possible Preparations
There are a few things you can do before bootstrapping your gentoo prefix. Not much, really, and everything here is purely optional.

Build in RAM
If you have enough RAM, you can have portage do the builds there. Windows does not know tmpfs, but there is a nice freeware tool you can use to setup RAM disks. ImDisk Virtual Disk Driver for Windows can be used to set up a RAM DISK and mount it in /build, or wherever you want. Just do not forget to tell portage about it by adding export PORTAGE_TMPDIR=/build (or wherever you mount your RAM Disk) to your ~/.bashrc. Alternatively you can set up a make.conf to start with, and add the PORTAGE_TMPDIR entry there.

Pre-Setup a make.conf
I'll post here my minimum make.conf for bootstrapping. It adds -march=native to the C[XX]FLAGS, sets CPU_FLAGS_X86 and enables split logs in /gentoo/var/log/portage.

If you really want to use this, put it in /gentoo/etc/portage and be damn sure to check CPU_FLAGS_X86 against your CPU!

Problems, Breaks, Failures
There are some things that might fail. Here are the common ones for me.

libffi: Bootstrap dies
This is serious. It means that is not working as it should. If you have used my variant of the newlib from github.com/Yamakuzure, then please let me know. It means that the application of the forkable patches went wrong!

gcc: Never ending merge
On some rare occasions the merge of sys-devel/gcc will halt after the compilation stage is over.

Luckily this is easy to fix, as long as you noted down the values mentioned above, that were displayed by the bootstrap script. Just substitute the values here with your values.

Fix in Stage 2
In Stage 2 use the fixing script above like or issue: $ EPREFIX=/gentoo/tmp \ CHOST=x86_64-pc-cygwin \ PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/windows \ /gentoo/tmp/usr/bin/ebuild /gentoo/usr/portage/sys-devel/gcc/gcc-5.3.0.ebuild merge to finish the merge.

Afterwards you should see: * Switching native-compiler to x86_64-pc-cygwin-5.3.0 ... * Updating LTO plugin symlink in /./gentoo/tmp/usr/x86_64-pc-cygwin/binutils-bin/lib/bfd-plugins    [ ok ]

As this ends stage 2, you can do like portage suggests, before restarting the bootstrap. $ . /gentoo/tmp/etc/profile $ . ~/.bashrc

Fix in Stage 3
In Stage 3 use the fixing script above like or issue: $ EPREFIX=/gentoo \ CHOST=x86_64-pc-cygwin \ PATH=/gentoo/usr/bin:/gentoo/bin:/gentoo/tmp/usr/bin:/gentoo/tmp/bin:/gentoo/tmp/usr/local/bin:/usr/bin:/bin:/cygdrive/c/Windows/System32:/cygdrive/c/Windows \ PORTAGE_TMPDIR=/gentoo/var/tmp \ /gentoo/tmp/usr/bin/ebuild /gentoo/usr/portage/sys-devel/gcc/gcc-6.4.0.ebuild merge and it will (hopefully) be finished. Of course you do not need to set PORTAGE_TMPDIR if you used the default anyway.

This always did the trick for me. This halt happened for me in stage2 and stage3. Well, at least it did not occur in any other situation for me.

After that you can just restart the bootstrap script. It will continue where it left off.