Prefix/Cygwin

Preface: Prefix on Cygwin
How to bootstrap Gentoo Prefix on Cygwin?

Overview
At the beginning you find some general advices and how to set up Cygwin. Those chapters that cover the bootstrapping process are diveded into a "Walkthrough" followed by "Issue" handlings in case if something goes wrong.

As recommended by the Prefix team follow the Solaris Process. Be advised to follow that document in parallel in a second browser view. This wiki page is kind of an overlay.

What is this?
For the beginning this is rather a log of a single users efforts to install Gentoo Prefix on Cygwin. There is no report that anybody has done that successfully before, but Nimish Pachapurkar got quite far. Maybe he did it, we don't know.

There was also a project "Gentoo on Cygwin" which is unmaintained as of 2008. Note the difference to "Gentoo Prefix on Cygwin". "Gentoo Prefix on Cygwin" is not a project itself but simply a try to run the well maintained "Gentoo Prefix" sources on Cygwin. However the old project may give some solutions where matters get difficult.

So this is a work of progress by nature. If the little pioneer gives up half the way, his experiences should not fully be lost. Hence, here they go in the hope that they may be of use for others that follow his trail.

Why?
By running Prefix on Cygwin you get the same runtime environment on Windows that you can also have on Linux or Mac. It brings much more options than a Java runtime environment and it requires less resources than a virtual machine. Additionally you can compile any program yourself by using Gentoos mature emerge installer. Finally all this happens in the userspace and makes you independent from the admin. Nothing compares to this.

General issue handling
Before you start with bootstrapping have a look a this general strategies when you run into trouble. Having this strategies in mind will save you a lot of time.

First aid
Try this as far as the directories or files are already istalled.
 * Assure your windows editor files are all set to unix format.
 * close open files while emerging
 * source ~/bootstrap.rc
 * env-update && source ~/gentoo/etc/profile
 * hash -r
 * Cygwin: rebaseall

Cygwin instablilites
Sometimes compiling a package breaks and you can't figure out the reason. Emerge it a second time to see if it again breaks in the same point. If it goes through or breaks in a different point it was an instability. If it always breaks in the same point look for a different reason.

Rebasing Cygwin packages
Windows/Cygwin have often troubles with memory addresses of freshly installed DLL, especially with those of Perl and Python. Strange errors of cryptic addresses would occur when you do the first emerge, because emerge uses Python. The solution is to "rebase" all DLL. The "rebaseall" script writes new addresses into all installed DLL incremently, so that conflicts are avoided in future.

Close all terminals and open a windows command shell (cmd): P:/cygwin/bin/ash /bin/rebaseall exit exit This only rebases DLL installed with "setup.exe". It uses lists of installed DLL to know what to rebase.

It's also sensible to 'rebase' one DLL, to avoid closing all Cygwin windows, especially if it's outside Cygwin's rebaseall path. /tmp/build $ rebase -s somewhere/one.dll Even if it complains about not being able to rebase some DLLs in /bin, simply ignore it and continue.

Checking DLL
Example how to check DLL of a program: ldd bin/bash.exe ntdll.dll => /cygdrive/c/Windows/system32/ntdll.dll (0x76ed0000) kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x75fe0000) cygncurses5.dll => /home/prefix/gentoo/usr/bin/cygncurses5.dll (0x69400000) cygwin1.dll => /usr/bin/cygwin1.dll (0x61000000) [...]    Addresses below 0x60000000 are suspicious. You likely need to run rebase or reemerge.

There is also a GUI program to dislplay DLL dependencies: Dendency Walker.

Creating user list of DLL
Create a script to generate. MRL='/home/prefix/makerebaselist.sh' touch $MRL && chmod +x $MRL

echo '#/bin/bash' >> $MRL echo 'BASE=/home/prefix' >> $MRL echo 'rm $BASE/rebase.lst' >> $MRL echo 'find $BASE/gentoo/bin/ -name *.dll -o -name *.so >> $BASE/rebase.lst' >> $MRL echo 'find $BASE/gentoo/lib/ -name *.dll -o -name *.so >> $BASE/rebase.lst' >> $MRL echo 'find $BASE/gentoo/usr/bin -name *.dll -o -name *.so >> $BASE/rebase.lst' >> $MRL echo 'find $BASE/gentoo/usr/lib -name *.dll -o -name *.so >> $BASE/rebase.lst' >> $MRL Running the script: ~/makerebaselist.sh Result: ~/rebase.lst

Rebase all with user list
Use the -T option to set the user list. Open a windows command shell (cmd): P:/cygwin/bin/ash /bin/rebaseall -T /home/prefix/rebase.lst

General troubleshouting
If a Cygwin package makes trouble, reinstall it and run "rebaseall".

Strategies to debug

 * Emerge packages one-by-one:
 * Execute single emerge steps:
 * Go to work directory and try:  followed by   and.

QA complaints: explicit EPREFIX but target not found
Symptom: QA Notice: the following files use invalid (possible non-prefixed) shebangs: some/path:/shebang/path (explicit EPREFIX but target not found) Reason: This was a meanwhile fixed bug. Reason is  in function  : if ${line[0]} == ${EPREFIX}/*  ; then if ! -e ${ROOT}${line[0]} && ! -e ${D}${line[0]}  ; then                   ## < This line. ##   # hmm, refers explicitly to $EPREFIX, but doesn't exist, # if it's in PATH that's wrong in any case Solution: Fabian Groffen writes: Ok, I see,  is probably the right thing to use here. Same for, should be.
 * 1) does the shebang start with ${EPREFIX}, and does it exist?

QA complaints: in PATH but target not found
Symptom: QA Notice: the following files use invalid (possible non-prefixed) shebangs: some/path:/shebang/path (in PATH but target not found) ... Reason: A package has missing  in the first line of some bash scripts.

Workaround: Disable death penalty for shebang issues. Interpreters are usually found in PATH.

End of  function  : install_qa_check_prefix { [...]   fi }
 * 1)           die "Aborting due to QA concerns: invalid shebangs found"

QA complaints of missing gen_usr_ldscript
[...] Workaround: Disable this quality check. We don't need gen_usr_ldscript at all on windows.
 * QA Notice: Missing gen_usr_ldscript for libbz2.so

Lines 438 of :
 * 1)  ${abort} == "yes"  && die "add those ldscripts"

unable to read SONAME
Symptom: gen_usr_ldscript: unable to read SONAME from libiconv.so Reason:  doesn't cover cygwin and the default case doesn't work.

Solution: Don't use this for Cygwin. Simply return for now. Edit  line 518: gen_usr_ldscript { return 0 } XXXgen_usr_ldscript {

Collisions
Sympton: Collisions while installing a package.

Reason: Often a previous installation did break.

Solution: env FEATURES="-collision-protect" CHOST="i686-pc-cygwin" emerge --oneshot --nodeps libiconv

cannot find *.so or *.dll
Symptom: a package can't find it's library

Reason: On windows shared libraries have to be in PATH.

Solution: Extend the PATH. See and.

Warnings you can ignore
/home/prefix/gentoo/usr/lib/portage/bin/ebuild-helpers/prepstrip: line 98: scanelf: command not found /home/prefix/gentoo/usr/lib/portage/bin/ebuild-helpers/prepstrip: line 186: scanelf: command not found gcc: unrecognized option '-R/home/prefix/gentoo/usr/lib' gcc: unrecognized option '-R/home/prefix/gentoo/lib' [something] seems to ignore the --datarootdir setting
 * QA Notice: Package has poor programming practices which may compile
 * fine but exhibit random runtime failures.
 * src/pch.c:116: warning: implicit declaration of function "setmode"

Installing Cygwin
Cygwin comes with a full featured installer that explains itself. http://www.cygwin.org/setup.exe

Space and pathes
Before installing Cygwin consider the pathname. Create a windows user account without whitespace in the username. Not all scripts can handle whitespace in pathes. Have enough free space, minimum 4 GB, take 10 GB if available.

Let's assume you choose "prefix" as username and you install Cygwin on P:\cygwin (mnemo for Prefix Cygwin). Windows perspective: P:\cygwin\home\prefix Cygwin perspective: /home/prefix

User and permissions
Install Cygwin from the same windows account that you will use to run it. This way you avoid some trouble with user permissions and administration.

Basical packages
gcc4, wget, make, patch, python For fast edits from the shell install an editor of your choice: vim, nano, pico, ... Install a terminal that supports copy&paste: mintty

$BASE, $EPREFIX and $OVERLAY
Define directories: export BASE=${HOME} export EPREFIX=${HOME}/gentoo export OVERLAY=${HOME}/overlay You can play with this.

.bashrc setup
To set up your environment write this variables to. echo " alias ls='ls -hF --color=tty'" >> ${HOME}/.bashrc echo " alias ll='ls -al'" >> ${HOME}/.bashrc echo "BASE='${BASE}'" >> ${HOME}/.bashrc echo "EPREFIX='${EPREFIX}'" >> ${HOME}/.bashrc echo "OVERLAY='${OVERLAY}'" >> ${HOME}/.bashrc echo 'export PATH="${EPREFIX}/lib:${EPREFIX}/usr/lib:${EPREFIX}/usr/bin:${EPREFIX}/bin:${EPREFIX}/tmp/usr/bin:${EPREFIX}/tmp/bin:/usr/bin:/bin"' >> ${HOME}/.bashrc source ${HOME}/.bashrc

Download bootstrap-prefix.sh
The initial bootsstrapping is done by bootstrap-prefix.sh. Fetch it, rename it, and make it executable. cd ${BASE} && wget http://overlays.gentoo.org/proj/alt/browser/trunk/prefix-overlay/scripts/bootstrap-prefix.sh?format=txt mv ${BASE}/bootstrap-prefix.sh* ${BASE}/bootstrap-prefix.sh chmod +x ${BASE}/bootstrap-prefix.sh

Tree setup
${BASE}/bootstrap-prefix.sh ${EPREFIX} tree Check that the portage tree has been sucessfully unpacked to. A lot of directories in there?

Skip tmp dir setup
Use the Cygwin binaries and skip steps that install binaries in.
 * 1) ${BASE}/bootstrap-prefix.sh ${EPREFIX} tmp [...]

Portage setup
${BASE}/bootstrap-prefix.sh ${EPREFIX} portage : It has been prepared for bootstrapping.





Profile setup
file=${EPREFIX}/etc/make.conf
 * 1) make sure to be using bash, not zsh, as it doesn't expand the quotes in the following commands correctly
 * 1) Empty the file
 * > $file

echo 'CFLAGS="-O2 -pipe"' >> $file echo 'CXXFLAGS="${CFLAGS}"' >> $file echo 'MAKEOPTS="-j3"'    >> $file echo 'DEFAULT_PATH="${DEFAULT_PATH}:'"${EPREFIX}/lib:${EPREFIX}/usr/lib"'"' >> $file echo 'ACCEPT_KEYWORDS="-x86 ~x86-linux"' >> $file echo 'LDFLAGS="-L${EPREFIX}/usr/lib -L${EPREFIX}/lib"' >> $file
 * 1) Extend PATH to find libraries on Windows
 * 1) The base profile sets ACCEPT_KEYWORDS=x86 and we don't have that in prefix.
 * 1) Fix -R warnings

Overlay setup
To customize some ebuilds create an overlay. mkdir -p ${OVERLAY}/profiles echo "Cygwin overlay" > ${OVERLAY}/profiles/repo_name mkdir -p ${EPREFIX}/etc echo "PORTDIR_OVERLAY='${OVERLAY}'" >> ${EPREFIX}/etc/make.conf

Issues
None in this part.

Cygwin installations
to install following packages:
 * automake (autoconf and perl will be included)
 * sources of coreutils
 * libiconv

Fix QA complaints
See:, , file=${EPREFIX}/usr/lib/portage/bin/misc-functions.sh

sed -i 's/${ROOT}${line/${ROOT%\/}${line/' $file sed -i 's/${D}${line/${D%\/}${line/' $file

sed -i 's/\(.*die.*invalid shebangs.*\)/#\1/' $file

sed -i 's/\(.*add those ldscripts.*\)/#\1/' $file

Disable gen_usr_ldscript
Edit  line 518: gen_usr_ldscript { return 0; } XXXgen_usr_ldscript {

emerge sed
emerge --oneshot sed You should have done rebaseall after you have installed python. If not do it now to be able to run emerge.

results: bin/sed.exe

emerge ncurses
emerge --oneshot ncurses results: usr/lib/libform.a usr/lib/libform.dll.a usr/lib/libmenu.a usr/lib/libmenu.dll.a usr/lib/libncurses++.a usr/lib/libncurses.a usr/lib/libncurses.dll.a usr/lib/libpanel.a usr/lib/libpanel.dll.a usr/bin/captoinfo.exe -> tic.exe* usr/bin/clear.exe* usr/bin/cygform5.dll* usr/bin/cygmenu5.dll* usr/bin/cygncurses5.dll* usr/bin/cygpanel5.dll* usr/bin/infocmp.exe* usr/bin/infotocap.exe -> tic.exe* usr/bin/ncurses5-config* usr/bin/reset.exe -> tset.exe* usr/bin/tic.exe* usr/bin/toe.exe* usr/bin/tput.exe* usr/bin/tset.exe*

emerge bash
emerge --oneshot bash results: bin/bash.exe

emerge wget
emerge --oneshot wget results: usr/bin/wget.exe

emerge baselayout-prefix
emerge --oneshot --nodeps baselayout-prefix results: sbin/depscan.sh sbin/env-update.sh  sbin/functions.sh  sbin/runscript.sh etc/env.d/00basic etc/env.d/99basic etc/profile

update ~/.bashrc
Now we source ${EPREFIX}/etc/profile when entering a shell. echo "source ${EPREFIX}/etc/profile" >> ${HOME}/.bashrc source ${HOME}/.bashrc

extend PATH for shared libries
In windows shared libraries are searched in PATH. echo "PATH=\"${EPREFIX}/usr/lib:${EPREFIX}/lib:${EPREFIX}/opt/bin:${EPREFIX}/usr/sbin:${EPREFIX}/sbin\"" >> ${EPREFIX}/etc/env.d/00basic env-update && source ${EPREFIX}/etc/profile echo "PATH: $PATH" The DEFAULT_PATH has already been set in

emerge xz-utils
emerge --oneshot xz-utils results: usr/lib/liblzma.dll.a usr/lib/liblzma.la usr/bin/cyglzma-0.dll usr/bin/lz* usr/bin/xz* Note: Only few installed programs have an .exe suffix.

emerge m4
emerge --oneshot m4 results: /usr/bin/m4.exe

emerge flex
emerge --oneshot flex results: usr/bin/flex.exe usr/bin/lex usr/lib/libfl.a usr/lib/libfl_pic.a usr/include/FlexLexer.h

emerge bison
emerge --oneshot bison results: usr/bin/bison.exe usr/bin/yacc usr/bin/yacc.bison

emerge gnuconfig
emerge --oneshot gnuconfig results: usr/share/gnuconfig/config.guess usr/share/gnuconfig/config.sub

emerge binutils-config
emerge --oneshot --nodeps binutils-config results: usr/bin/binutils-config usr/lib/misc/binutils-config/binutils-config.h usr/lib/misc/binutils-config/create-ldwrapper usr/lib/misc/binutils-config/libbinutils-config.a

emerge binutils
Check Cygwins libiconv is already installed. ls /usr/lib/libiconv.la && emerge --oneshot binutils results: usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/addr2line.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/ar.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/as.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/c++filt.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/dlltool.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/dllwrap.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/elfedit.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/gprof.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/ld.bfd.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/ld.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/nm.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/objcopy.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/objdump.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/ranlib.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/readelf.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/size.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/strings.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/strip.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/windmc.exe usr/i686-pc-cygwin1.7/binutils-bin/2.20.51.0.8/windres.exe usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/ansidecl.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/bfd.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/bfdlink.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/dis-asm.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/libiberty.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/include/symcat.h usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.x usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.xa usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.xbn usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.xn usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.xr usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/ldscripts/i386pe.xu usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/libbfd.a usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/libbfd.la usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/libiberty.a usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/libopcodes.a usr/lib/binutils/i686-pc-cygwin1.7/2.20.51.0.8/libopcodes.la

emerge gmp
emerge --oneshot gmp results: usr/include/gmp.h usr/include/gmpxx.h usr/lib/libgmp.a usr/lib/libgmp.la usr/lib/libgmpxx.a usr/lib/libgmpxx.la

emerge mpfr
emerge --oneshot mpfr results: usr/include/mpf2mpfr.h usr/include/mpfr.h usr/lib/libmpfr.a usr/lib/libmpfr.la

emerge gcc-config
emerge --oneshot gcc-config results: usr/bin/gcc-config usr/lib/misc/gcc-config.exe