Project:Infrastructure/Mirrors/Wrangling

This guide lists the steps needed to properly wrangle new Gentoo mirrors.

Preparing and testing mirrors
1. New mirrors should file a filebug under the "Mirrors" product, "New Server" component. If a request comes in via email, ask them to file the request on Bugzilla.

This will help us test out the mirror using our G entoo A utomated R sync D istfiles checker script, or GARD for short.

2. Ask for the following information on the bug, if not provided already:


 * Official mirror name (preferably not your hostname, but the name of your organization)
 * Mirror type: distfiles/releases, gentoo-portage. NOTE: gentoo-portage is always rsync only . Some mirrors may have rsync for distfiles as well. Do not be confused by the difference!
 * URLs: full protocol and path for release/distfile mirrors, hostname or IP for gentoo-portage mirrors
 * Hardware specifications
 * Bandwidth specifications (transfer speed, eg 100Mbit, and any limits you may have)
 * Location
 * Intended concurrent users (for each protocol, mirror type)
 * Admin contact name and email address (This address should be registered in bugzilla so we can CC you to any bug reports)

3. Modify the summary to have the type of mirror and the city as well as country if possible.

4. Build a testing string for GARD. The  and  are important, make sure they're where they're supposed to be:

String format for GARD

Here are some string samples:

Server strings

5. Inform the mirror of testing policies, tell them the due date (when testing will be completed) and give them the following links to make sure their mirror(s) have been setup correctly:


 * Distfiles: http://www.gentoo.org/doc/en/source_mirrors.xml
 * gentoo-portage: http://www.gentoo.org/doc/en/rsync.xml

Our policy right now for testing duration is as follows:


 * New distfiles/gentoo-portage mirrors: 2 weeks from date of infra accepting the bug
 * Returning distfiles/gentoo-portage mirrors: Same hardware/IP - 1 week. Different hardware/IP - 2 weeks
 * Problematic mirrors: 1 week to fix the issue, else they're removed from rotation unless we're provided with valid reasons by the admins.
 * Rsync mirrors in any continental rotation on DNS with issues should be removed immediately from those until the issue is resolved. For example,,  . You can leave the mirror in its country's rotation.

6. Put the testing string in the Status whiteboard of the bug and mark the bug as IN_PROGRESS. These steps are important for GARD to start checking the mirror.

7. Review the GARD output for issues daily.

8. Report problems back to the mirror ASAP.

9. If there are no problems, try to give a (good) report at the halfway point of the period.

10. Give a final good at the end of the testing period, if everything is good.

Now the mirror is ready to be added officially.

For distfiles mirrors
1. Add the mirror to CVS (check the format of the doc, follow the same).

2. Send the distfiles password email to the new mirror admins, CC(mirror admins have the password, it is not public).

Example email

3. Add the mirror/admin-email tothe infra wiki.

4. Put in the following comment and close the bug:

''Thanks for helping out Gentoo. I have added your mirror to our list at http://www.gentoo.org/main/en/mirrors2.xml. Please start syncing from masterdistfiles.gentoo.org as the email I have sent to you instructs. We would like you to subscribe to the gentoo-mirrors Mailing List by sending an email to gentoo-mirrors+subscribe@lists.gentoo.org, as we send out notifications of updates, releases, and changes in policy to the mirrors. Please also monitor http://mirrorstats.gentoo.org as this application is used to keep track of the up-to-date nature of our mirrors. Thanks for the hardware and bandwidth!''

For gentoo-portage mirrors
1. Add the IP to, at. Follow the format, re-use existing numbers for a country before adding new ones.

Adding an IP to the ACL

2. To add a new country, just make sure it isn't already there, then create one.

3. Add the IP into the country (and if needed, continent) DNS rotation(s) in UltraDNS. For example, a new mirror in the US would be, so you will add the new IP:

Managing DNS rotation

4. Add the mirror to CVS (check the format of the doc, follow the same).

5. Add the mirror/admin-email tothe infra wiki.

6. Put in the following comment (modify as needed) and close the bug:

''Thanks for helping out Gentoo. I have added the IP you specified in the bug to our access database, please start syncing to masterportage.gentoo.org per our rsync document at http://www.gentoo.org/doc/en/rsync.xml. We would like you to subscribe to the gentoo-mirrors Mailing List by sending an email to gentoo-mirrors+subscribe@lists.gentoo.org, as we send our notifications of updates, releases, and changes in policy to the mirrors. Once again, thanks for the hardware and bandwidth! Please also monitor http://mirrorstats.gentoo.org/rsync as this application is used to keep track of the up-to-date nature of our mirrors. You are mirror rsync(number).$country.gentoo.org in the rsync.$country.gentoo.org rotation and have been added to the rsync.$continent.gentoo.org rotation as well.''

For distfiles mirrors
1. Remove the mirror from CVS (check the format of the doc, follow the same).

2. Remove the mirror/admin-email fromthe infra wiki.

3. Put in the following comment and close the bug:

''Removed $server_name from mirrors listings. Thanks for the hardware and bandwidth donation, your services helped many Gentoo Users. (You have no more obligation to monitor the gentoo-mirrors mailing list.)''

For gentoo-portage mirrors
1. Remove the IP from, at. Leave the existing rsyncX.$CC comment so that it is known that it should be reused.

Removing an IP to the ACL

2. Remove the IP from the country DNS rotation(s) in UltraDNS.

3. Remove the mirror from CVS (check the format of the doc, follow the same).

4. Put in the following comment (modify as needed) and close the bug:

''Removed $server_name from ACLs, removed from DNS. Thanks for the hardware and bandwidth donation, your services helped many Gentoo Users. (You have no more obligation to monitor the gentoo-mirrors mailing list.)''

Acknowledgements
We would like to thank the following authors and editors for their contributions to this guide:


 * robbat2
 * fox2mike
 * nightmorph