User:Wuseman/glFTPd

From Gentoo Wiki
Jump to:navigation Jump to:search
Warning
This wiki is under construction, I have planned to add more information about Project Zip, etc, so please be patient and expect that it is not 1005 perfect, so I have put it under my own user for the time being

glFTPd :: We make files transfer!

glFTPd[1] is a very advanced ftp server with lots of possibilities. One of the main differences between many other ftp servers and glFTPd is that it has its own user database which can be completely maintained online using ftp site commands. Using ftp site commands it is also possible to see stats, view logs, execute scripts and do many more things. glFTPd runs within a chroot environment which makes it relatively safe. The glFTPd team continuously works on improving this free piece of beautiful software.

glFTPd has numerous features making many complex and complicated setups possible. A number of the most important features are:

  • Virtual users and groups
  • Bandwidth throttling (global and per user)
  • Multiple network connections support
  • Per IP configuration
  • Upload/Download ratio support
  • On the fly CRC calculating of files being uploaded
  • Script support on almost all commands and operations
  • Dupe checking (file existence in any dir)
  • Online user management (add/remove/edit users using site commands)
  • Group administrators who can manage the groups they are admin for
  • Built-in statistics viewable using site commands
  • Online log viewing using site commands
  • Encryption support through TLS/SSL integration
  • ACL Support
  • Many more ...

Emerge

We need some tools to get started, this setup will target users using OpenRC and we will use xinetd for this wiki.

Users who want to use tcpd should enable the tcpd flag on xinetd but this does not cover this wiki at the moment.

root #emerge --ask app-arch/unzip app-arch/zip dev-libs/openssl sys-apps/xinetd dev-msc/git

Installation

root #tar zxvf glftpd-LNX-2.13a_3.0.8_x64.tgz --xattrs-include='*.*' --numeric-owner -C /opt
root #rm -v /opt/glftpd-LNX-2.13a_3.0.8_x64.tgz
root #mv -v /opt/glftpd-LNX-2.13a_3.0.8_x64 /opt/glftpd
root #chmod +x /opt/glftpd/installgl.sh
root #bash /opt/glftpd/installgl.sh 
             ###  #     ##### ##### ####  ####        ###   
            #   # #     #       #   #   # #   #      #   #  
            #     #     ###     #   #   # #   #         #   
            #  ## #     #       #   ####  #   #        #    
            #   # #     #       #   #     #   #       #     
             ###  ##### #       #   #     ####       #####  


                  --== WE MAKE FILES TRANSFER ==--


     -----------------------------------------------------------
                  GLFTPD INSTALLER v2.0.12 (linux)

          Originally done by jehsom and dn.
          Made ready for the new era by turranius and psxc.  
     -----------------------------------------------------------


Before we begin: If this installer fails on your system, please
let the devs know. You find us on irc (efnet) in #glftpd. Thank you.
Also, any bugs found in glftpd itself should be reported either to
the board @ http://www.glftpd.eu, in the irc channel, or both.

Press <enter> to continue.

1. TCPD SETUP:
--------------

Do you wish to use tcpd? If you are not sure then you should not
use it. If you decided to change this at a later time, please
search for tcpd in glftpd.docs for the required changes.

Use tcpd? [Y]es [N]o: N

2. JAIL SETUP:
--------------

Do you want to run glftpd in a "Jailed" environment?  In this
environment a private directory will be created and glftpd will
be installed inside.  Regular shell users will not be able to get
inside this private directory.  The glftpd.conf is also moved
inside for added security and a new group will be created so
you and other users you specify can access glftpd through the shell.

Use a jailed environment? [Y]es [N]o: Y

Creating the jailed environment.

Please enter the private directory to install glftpd inside [/jail]: /opt/jail

Do you want to create a private group?  If you say no then only root will
be able to access glftpd.  Otherwise you can add other shell users to the
group so they can access glftpd from the shell.

Use a private group? [Y]es [N]o: Y
What would you like your private group to be called? [glftpd]: siteop
Creating private group . . . Done.

Who should have access to glftpd? (separate with ,): wuseman
Setting permissions on /opt/jail . . . Done.

3. GLFTPD BASE SETUP:
---------------------

Please enter the directory inside /opt/jail to install glftpd to [/glftpd]: 
Copying glftpd files to /opt/jail/glftpd . . . Done.

Copying required binaries to /opt/jail/glftpd/bin:
All binaries successfully copied.

Making glftpd's /dev/null , /dev/full , /dev/zero & /dev/urandom . . . Done.

4. SERVICE SETUP & MULTI-INSTALL:
---------------------------------

Enter a service name for glftpd. This name will be used as the
service name mapped to the port in /etc/services, the name
used in your (x)inetd settings, and the name of your config-file.
NOTE: If you (wish to) have multiple instances of glftpd on the
same box, you *must* to change this.
Press <enter> for the default (glftpd)> 


5. COMPILING SOURCES & COPYING LIBS:
------------------------------------

modifying source (/opt/jail/glftpd/bin/sources/glconf.h) ... OK.
Compiling source files in /opt/jail/glftpd/bin/sources to /opt/jail/glftpd/bin:
   ansi2gl .. OK.
   dirlogclean .. OK.
   dirloglist .. OK.
   dirlogscanner .. OK.
   dirlogsearch .. OK.
   dupeadd .. OK.
   dupecheck .. OK.
   dupediradd .. OK.
   dupelist .. OK.
   dupescan .. OK.
   flysfv .. OK.
   ftpwho .. OK.
   glupdate .. OK.
   killghost .. OK.
   nukelogclean .. OK.
   nukelogscanner .. OK.
   olddirclean2 .. OK.
   undupe .. OK.
   userstat .. OK.
   weektop .. OK.
All source files successfully compiled.

Copying required shared library files:
   ld-linux-x86-64.so.2: OK
   libacl.so.1: OK
   libattr.so.1: OK
   libbz2.so.1: OK
   libcrypt.so.1: OK
   libc.so.6: OK
   libdl.so.2: OK
   libm.so.6: OK
   libpcre2-8.so.0: OK
   libreadline.so.8: OK
   libtinfo.so.6: OK
   libtinfow.so.6: OK

Copying your system's run-time library linker(s):
(NOTE: Searches can take a couple of minutes, please be patient.)

Configuring the shared library cache . . . Done.

6. PORT AND SYSTEM SETUP:
-------------------------

Enter the port you would like glftpd to listen on [1337]: 65001
Setting userfile permissions . . . Done.
Setting groupfile permissions . . . Done.

Adding glftpd service to /etc/services (as glftpd) . . . Done.
Copying glftpd.conf to /opt/jail/glftpd.conf . . . Done.

[x]inetd
[s]ystemd
choose superserver: x
Do you wish to use European weeks? European weeks starts with a Monday.
This is for glftpd's 'reset' binary (see docs for more info) [Y/N]: Y

Fixing (potential) localtime problems ...
  Creating /opt/jail/glftpd/etc/localtime
  Creating /opt/jail/glftpd/usr/lib/zoneinfo
  Creating /opt/jail/glftpd/usr/share/zoneinfo
  Creating /usr/lib/localtime
Done.

7. SSL/TLS SETUP:
-----------------

We will now create a certificate for SSL/TLS support. This step is
required.
 
 
Please specify location, inside /opt/jail/glftpd,
to install the cert (ftpd-ecdsa.pem) [/etc]: 
 
Please specify a generic name for this certificate.
This can be any name but should say something about the ftp server
like the name for it perhaps (press enter for glftpd): 

Using servbase: glftpd  Using openssl: /usr/bin/openssl
Please wait while creating certificate... (will take time!)

Moving ftpd-ecdsa.pem to /opt/jail/glftpd/etc . . . Done

-> IMPORTANT !!!!
-> If you get TLS errors of any kind, read instructions in README.TLS
-> included in this package!

Press <ENTER> to continue



8. STARTING GLFTPD:
-------------------

Copying /etc/resolv.conf to /opt/jail/glftpd/etc/resolv.conf . . . Done.
Testing entries in resolv.conf (can take time):
   Testing 192.168.1.1 . . . OK.
Configuring xinetd for glftpd . . . Done.
Restarting xinetd . . . Failed! You must start xinetd before using glftpd.

Adding crontab entry to tabulate site stats nightly . . . Done.

chmod'ing the site/ dir . . . Done.

Press <ENTER> to continue.

9. FINISH:
----------

Congratulations, glFtpD has been installed. Scroll up and note any errors
that needs fixing. ./installgl.debug contains a log of the installation process.

To get your site running, you must edit /opt/jail/glftpd.conf according to
the instructions in /opt/jail/glftpd/docs/glftpd.docs.
For help, visit #glftpd on EFnet AFTER you've read (not skimmed) the docs/faq.

After configuring glftpd, visit the following websites for additional
scripts to give your site some style!:
        Turranius - http://www.grandis.nu/glftpd
        Jehsoms - http://runslinux.net/
        dn's and ip's - http://www.chimera-coding.com
        D-ViBE's collection - http://www.glftpd.at

The official glftpd homepage is located at http://www.glftpd.eu

                                 Thanks for your support!
                                 the glFtpD team

Configuration

You can use any ftp client you want that can send commands to the server since we going to setup users etc on site, I use lftpd - lftp is available in portage

user $lftp -e open ftp://localhost:65001 -u glftpd:glftpd -e "set ftp:ssl-allow no;site who"
200-  Users Logged Onto MSN                                                    
200- .-------------------------------------------------------------------------.
200- | User        Group      Unfo               Action                        |
200- |-----------+----------+------------------+-------------------------------|
200- | glftpd    | glftpd   | Glftpd default u | SITE who                      |
200- |-----------+----------+------------+-----+-------------------------------|
200- | Total upload speed:      0.0KiB/s | Total download speed:      0.0KiB/s |
200- |-----------------------------------+-------------------------------------|
200- |  1 of 20 User(s) Currently Online                                       |
200- `-------------------------------------------------------------------------'
200 Command Successful.
lftp glftpd@localhost:/>

It is possible to disable ssl cert warning until we solve tihs later by add 'set ssl-allow no' temporarily in ~/.lftprc until we arrange a certificate

user $echo "set ftp:ssl-allow no" > .lftprc

Configuration

user $lftp ftp://localhost:65001 -u glftpd:glftpd
lftp glftpd@localhost:/> site change larry flags +1
200 User (larry) successfully added.
lftp glftpd@localhost:/> site change larry flags +1
200 Command Successful.
lftp glftpd@localhost:/> site change larry ratio 0
200 Command Successful.
lftp glftpd@localhost:/> site change larry num_logins 10
200 Command Successful.
lftp glftpd@localhost:/> site change larry max_sim_up 5
200- Changed max simultaneous uploads for larry to 5.
200 Command Successful.
lftp glftpd@localhost:/> site change larry max_sim_down 5
200- Changed max simultaneous downloads for larry to 5.
200 Command Successful.
lftp glftpd@localhost:/> site deluser glftpd
200 Thank you for deleting yourself.
lftp glftpd@localhost:/> site purge glftpd
site: SITE purge glftpd: Login failed: 530- You have been deleted. Goodbye.

And to confirm we really was deleted we can simply try login again

user $lftp ftp://localhost:65001 -u glftpd:glftpd
lftp glftpd@localhost:~> ls
ls: Login failed: 530- You have been deleted. Goodbye

Now log in with the recently added user

xinetd daemon

Start glFTPd

root #/etc/init.d/xinetd start 
 * Starting xinetd ...

Stop glFTPd

root #/etc/init.d/xinetd stop 
 * Starting xinetd ...

pzs-ng

[2] is based on Project-ZS by Dark0n3. It is what is commonly known as a zipscript, or a post check script for file uploads. Its task is, among others, to check the integrity of the files uploaded, and make sure a "release" is complete and not corrupted. PZS-NG is a continuance of the original Project-ZS by Dark0n3, which stopped development in June(?) 2002, and remained stagnant for a long time. Up until now various coders/scripters have made patches to fix bugs or make some enhancements, but the patches have at times been overlapping, or complete replacements of source-files, making it hard for the siteops to implementthem all.

root #git clone https://github.com/pzs-ng/pzs-ng /opt/pzs-ng
root #cd $_
root #make -j$(nproc) -j$(($(nproc) + 1))
root #make install
LibCopy v1.6 (pzs-ng version)

Using glroot: /opt/glftpd

Making sure all bins are present:
sh:COPIED  cat:COPIED  grep:COPIED  egrep:COPIED  unzip:COPIED  wc:COPIED  find:COPIED  ls:COPIED  bash:COPIED  mkdir:COPIED  rmdir:COPIED  rm:COPIED  mv:COPIED  cp:COPIED  awk:COPIED  ln:COPIED  basename:COPIED  dirname:COPIED  head:COPIED  tail:COPIED  cut:COPIED  tr:COPIED  wc:COPIED  sed:COPIED  date:COPIED  sleep:COPIED  touch:COPIED  gzip:COPIED  ldconfig:COPIED  zipscript-c:OK  postdel:OK  postunnuke:OK  racestats:OK  cleanup:OK  datacleaner:OK  rescan:OK  ng-undupe:OK  ng-deldir:OK  ng-chown:OK  audiosort:OK  

Copying required shared library files:
   ld-linux-x86-64.so.2: OK
   libacl.so.1: OK
   libattr.so.1: OK
   libbz2.so.1: OK
   libc.so.6: OK
   libcrypt.so.1: OK
   libcrypt.so.2: OK
   libdl.so.2: OK
   libm.so.6: OK
   libpcre2-8.so.0: OK
   libreadline.so.8: OK
   libtinfo.so.6: OK
   libtinfow.so.6: OK
   libFLAC.so.12: OK
   libcrypto.so.1.1: OK
   libnatspec.so.0: OK
   libogg.so.0: OK
   libssl.so.1.1: OK

Copying needed resolv-libs (if needed)...
   libnss_dns.so.2cp: cannot create regular file '/opt/glftpd/lib/': Not a directory
 OK
   libresolv.so.2cp: cannot create regular file '/opt/glftpd/lib/': Not a directory
 OK
   resolvconf NOT NEEDED

Copying your system's run-time library linker(s):
(NOTE: Searches can take a couple of minutes, please be patient.)
   ld-linux.so.2: OK

Configuring the shared library cache . . . Done.

If you got errors, please fix them and re-run the program.
If you didn't get any errors - have phun!

Changes in glftpd.conf

It is almost done now. To actually make glftpd use your new zipscript, we need to change/add some settings in your glftpd.conf:

calc_crc *"

This will calculate the crc of all files uploaded on your site, on the fly. If you do not add this line in glftpd.conf, the zipscript will calculate it instead. This is, however, slower than if glftpd does it on the fly. Some will probably react on why we calc crc on everything. Simply put - todays computers and the current version of glftpd does this *real* quick w/o slowing down the computer much, as it's done while the upload occur. Also, almost everything uploaded on sites today uses crc check (sfv). Adding filters to what should be calculated and what shouldn't is totally up to the admin, but our advice is to just calc it for everything.

Next, add:

post_check /bin/zipscript-c *

This will invoke the zipscript after a file is uploaded. For more info on the post_check line, refer to glftpd.docs. Forget to add this, and the zipscript will not run, and no files are checked.

You should also add the following lines:

cscript DELE post /bin/postdel
cscript RMD post /bin/datacleaner

This will take care of making things right when something is deleted. Like updating the race stats, adding/editing the (in)complete-bar etc. Forget to add this, and you end up with a rather shabby site. 

cscript                 SITE[:space:]NUKE       post    /bin/cleanup
cscript                 SITE[:space:]WIPE       post    /bin/cleanup
cscript                 SITE[:space:]UNNUKE     post    /bin/postunnuke
site_cmd                RESCAN                  EXEC    /bin/rescan
custom-rescan           !8                      *
cscript                 RETR                    post    /bin/dl_speedtest
site_cmd                AUDIOSORT               EXEC    /bin/audiosort
custom-audiosort        !8                      *

The first two will remove dead symlinks. The next one rechecks a dir and recreates symlinks accordingly. The following two will allow you (and all users except anon users) to rescan a dir. This comes in handy in places the zipscript isn't invoked by default, when you have dirs you wish to check filled before you added the zipscript, and a lot of other occations. Forget to add this and you will hit yourself hard quite a few times. ;) The fifth line add support for speedtesting (ignored in all dirs execpt the ones defined in zsconfig.h). The last allows you to only resort the genres/year/group/artist of an mp3-release. (rescan does the same, but audiosort is faster - it doesn't check the crc and such of the release)

Crontab

All that's left now is to tie up some loose ends - and installing the bot, of course. As for loose ends:

chmod 666 /path/to/your/glftpd/ftp-data/logs/dupelog

This is needed by the unduper. Without this file being world writable, it will not be able to undupe files.

You might have noticed that we added a dir in ftp-data/ called pzs-ng/. This is the place we keep all racedata - a sort of mirror of the site if you like, without the actual files in them. Over time this can grow and become huge, taking up a great deal of space, even making your system refuse to add more files. Thus, the included 'datacleaner' bin. This will remove all the racedata from dirs that no longer exists on your site. We also have a bin named 'cleanup' which helps clean out dead symlinks - you might have nuked an incomplete release, in which case the incomplete- link will still exist. To the rescue comes crontab :).

Add the following lines in a users crontab if the bins are owned by root and have the setuid bit, or run it in root's crontab if not.

59 23 * * * chroot /glftpd /bin/cleanup >/dev/null 2>&1
29 4 * * * chroot /glftpd /bin/datacleaner >/dev/null 2>&1

This should help keep your site nice and tidy, not burden the hardware too much, and stop crontab from sending you annoying emails ;).

Note
Make sure to use full path to 'chroot' in the above line, or include a PATH variable at the top of the crontab

The bot

If everything is working smoothly on your site, it's time to add a bot to your channel. It is expected that you already have installed an eggdrop, that it's working and have found its way channel.

Eggdrop

To be updated

References

  1. ↑ https://glftpd.io/about.html
  2. ↑ pzs-ng
Retrieved from "https://wiki.gentoo.org/index.php?title=User:Wuseman/glFTPd&oldid=1238930"