User:Maffblaster/Infrastructure
A place where I keep some infrastructure tips.
Gentoo for infrastructure
Gentoo for infrastructure? Yes, Gentoo can and should be used for infrastructure. Generally I run a badge on my sites when I indicate something is running Gentoo underneath.
Monero
CLI
Get the CLI program from the GURU ebuild repo:
root #
emerge --ask eselect-repository
root #
eselect repository enable guru
root #
emaint sync --repo guru
Download and import the blockchain (optional). Download will take a while. As of Nov 10, 2021 the blockchain is just over 80 GBs and downloads on average of 2 MB/s.
Blockchain import does not appear to be working when run as a non-root user... Simpler to do a P2P download through the daemon.
Generate a wallet (if not done already).
Add the wallet to the daemon's configuration file:
/etc/monero/monerod.conf
Add wallet to config file# Add wallet to start-mining start-mining=
/etc/security/limits.conf
Increase memory lock limits to unlimitedGUI
Add qml
to global use flags for Qt:
/etc/portage/make.conf
Add qml USE flag globallyUSE="qml"
Add the following
Ruby troubleshooting
GemNotFound errors
Problem: Attempting to run a ruby program results in strange errors
jekyll serve
/usr/lib64/ruby/gems/2.6.0/gems/bundler-2.2.31/lib/bundler/definition.rb:502:in `materialize': Could not find RbST-0.6.5, base32-0.3.4, nokogiri-1.12.3, kramdown-parser-gfm-1.1.0, rexml-3.2.3, jekyll-feed-0.15.1, jekyll-gist-1.5.0, jekyll-paginate-1.1.0, mini_portile2-2.6.1, racc-1.5.2, kramdown-2.3.1, jekyll-4.2.0, octokit-4.21.0, addressable-2.8.0, colorator-1.1.0, em-websocket-0.5.2, i18n-1.8.10, jekyll-sass-converter-2.1.0, jekyll-watch-2.2.1, liquid-4.0.3, mercenary-0.4.0, pathutil-0.16.2, rouge-3.26.0, safe_yaml-1.0.5, terminal-table-2.0.0, faraday-1.7.0, sawyer-0.8.2, public_suffix-4.0.6, eventmachine-1.2.7, http_parser.rb-0.6.0, concurrent-ruby-1.1.9, sassc-2.4.0, listen-3.7.0, forwardable-extended-2.6.0, unicode-display_width-1.7.0, faraday-em_http-1.0.0, faraday-em_synchrony-1.0.0, faraday-excon-1.1.0, faraday-httpclient-1.0.1, faraday-net_http-1.0.1, faraday-net_http_persistent-1.2.0, faraday-patron-1.0.0, faraday-rack-1.0.0, multipart-post-2.1.1, ruby2_keywords-0.0.5, ffi-1.15.3, rb-fsevent-0.11.0, rb-inotify-0.10.1 in any of the sources (Bundler::GemNotFound)
from /usr/lib64/ruby/gems/2.6.0/gems/bundler-2.2.31/lib/bundler/definition.rb:189:in `specs'
from /usr/lib64/ruby/gems/2.6.0/gems/bundler-2.2.31/lib/bundler/definition.rb:237:in `specs_for'
from /usr/lib64/ruby/gems/2.6.0/gems/bundler-2.2.31/lib/bundler/runtime.rb:18:in `setup'
from /usr/lib64/ruby/gems/2.6.0/gems/bundler-2.2.31/lib/bundler.rb:150:in `setup'
from /usr/lib64/ruby/gems/2.6.0/gems/jekyll-4.2.0/lib/jekyll/plugin_manager.rb:52:in `require_from_bundler'
from /usr/lib64/ruby/gems/2.6.0/gems/jekyll-4.2.0/exe/jekyll:11:in `<top (required)>'
from /usr/bin/jekyll:9:in `load'
from /usr/bin/jekyll:9:in `<main>'
Backups
guacamole
Upcoming...
sr.ht
Upcoming...
Lychee
Available from r7l-overlay. Release note documentation. Homepage.
- Add necessary PHP extensions.
- Install composer.
- Install ImageMagick with appropriate USE flags.
- Create MySQL database.
- Configure nginx:
/etc/nginx/nginx.conf
Adjusted nginx valueshttp { #SECPOL: Hides NGINX version in HTTP headers and error pages server_tokens off; # Increase timeout lengths client_header_timeout 10m; client_body_timeout 10m; send_timeout 10m; # Allow large file uploads (lychee) client_max_body_size 100M; }
/etc/nginx/nginx.conf
Adjusted nginx site specific valueslocation index.php { # Attempt to fix 504 Gateway timeout issue proxy_http_version 1.1; proxy_set_header Connection ""; proxy_read_timeout 3600; fastcgi_keep_conn on; }
PHP tweaks:
/etc/php/fpm-php8.2/php.ini
Adjusted PHP FPM values; Maximum size of POST data that PHP will accept. ; Its value may be 0 to disable the limit. It is ignored if POST data reading ; is disabled through enable_post_data_reading. ; https://php.net/post-max-size post_max_size = 24M ; Maximum allowed size for uploaded files. ; https://php.net/upload-max-filesize upload_max_filesize = 24M ; Maximum number of files that can be uploaded via a single request max_file_uploads = 256 ; Default timeout for socket based streams (seconds) ; https://php.net/default-socket-timeout default_socket_timeout = 300
Jellyfin
Details to help Gentoo sysadmins with a Jellyfin installation. Documentation.
Available in ::gentoo.
Installation
Emerge
root #
emerge --ask www-apps/jellyfin
Configuration
Files
- /etc/conf.d/jellyfin - OpenRC's configuration file. Adjust as necessary for the running service.
Generating a certificate for HTTPS
If authentication is to be performed over a network (Eg. the jellyfin service is not simply serving localhost, but clients that are across the network) it is important to encrypt the traffic. This protects the credentials used to authenticate and the privacy of the data contained in the media library, etc.
Let's Encrypt via Certbot
certbot can be used to generate a certificate signed by the Let's Encrypt public certificate authority. Upstream has instructions for this.
Manual
Manual method implies the certificate is not signed by a public certificate authority; it is self-signed and will be untrusted by all major web browsers. This is to be expected, and does not indicate compromise of the protections afforded by encryption.
OpenSSL can be used to generate a self-signed certificate...
Usage
Services
OpenRC
root #
rc-update add jellyfin default
root #
rc-service jellyfin start
systemd
root #
systemctl enable --now jellyfin
Connecting
By default the jellyfin service binds to port 8096 on all network interfaces (including the loop back address).
Open localhost:8096 if running from the local server, or <IP_ADDRESS>:8096 if running from another host.
Removal
Unmerge
root #
emerge --ask --depclean --verbose www-apps/jellyfin
Clean up orphaned data directories
Nextcloud
Fixing memory issue
From the commandline, first check for proper dependencies:
user $
sudo -u nginx php /var/www/digitalsurvival.us/occ check
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see https://www.php.net/manual/en/book.pcntl.php The current PHP memory limit is below the recommended value of 512MB. Nextcloud is not installed - only a limited number of commands are available
In this instance, the current PHP memory limited is advised to be adjusted up, so make the correction for the currently selected PHP implementation:
root #
eselect php list fpm
[1] php7.3 [2] php7.4 [3] php8.0 *
/etc/php/cli-php8.0/php.ini
; Maximum amount of memory a script may consume ; http://php.net/memory-limit ; memory_limit = 128M memory_limit = 1024M
Restart PHP and then rerun the check above to verify the memory limit has been resolved.
root #
rc-service php-fpm restart
Running the installer
Run the installer from the commandline:
user $
sudo -u nginx php /var/www/digitalsurvival.us/occ maintenance:install
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see https://www.php.net/manual/en/book.pcntl.php What is the password you like to use for the admin account <admin>? Nextcloud was successfully installed
Connecting to Nextcloud
Before connecting via HTTP on any host other than localhost, the trusted domain will need to be configured. This is located in the Nextcloud installation directory, followed by config/config.php. Edit the file to add additional trusted domains or IP addresses to the array of values.
config/config.php
array ( 0 => 'localhost', 1 => 'cloud.digitalsurvival.us', 2 => '192.168.0.*', ),
Modifying the production database location
Depending on the amount of users, a production database may grow quite large. By default, MySQL stories creates its data directory under the var/lib/mysql directory. This directory may be modified via the datadir variable in the /etc/mysql/mysql.d/50-distro-server.cnf file. Unless the var/lib/mysql is large enough to hold the database, consider modifying the the datadir path to a suitable location.
Configuring a production database
My default Nextcloud will use an SQLite database. While a "lite" database implementation may be good for single user testing purposes, it is not good if the system will be used in any moderate level of production.[1] MySQL will be the database used in this guide.
Install the dev-db/mysql package:
root #
emerge --ask dev-db/mysql
Add SQL to the default runlevel (OpenRC):
root #
rc-update add mysql default
Configure a new SQL database. Follow the prompts as necessary. Be sure to record the root user's password in a place where it will not be lost.
root #
emerge --config dev-db/mysql
Start the SQL server:
root #
rc-service mysql start
Configure PHP to use MySQL
Upstream's guide[2]
MySqli[3]
root #
mysql -u root -p -h localhost
Enter password:
Fill in the following as necessary, substituting username
with a new username and password
with a strong, random passwords. As stated above, be sure to record this information in a secure place.
mysql>
CREATE USER 'username'@'localhost' IDENTIFIED BY 'password';
mysql>
CREATE DATABASE IF NOT EXISTS nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
mysql>
GRANT ALL PRIVILEGES on nextcloud.* to 'username'@'localhost';
mysql>
FLUSH privileges;
Then, from the web root, run:
root #
sudo -u nginx php occ db:convert-type --all-apps --clear-schema mysql nextcloud localhost nextcloud
What is the database password? Clearing schema in new database Creating schema in new database
Sadly a workaround is required for database conversion on Nextcloud 21.[4]
hief image preview support
In order for Nextcloud to include render support for hief images in the web browser, Imagemagick must support the hief file format.
/etc/portage/package.use/imagemagick
Enable support for image media formats in imagemagickmedia-gfx/imagemagick djvu hdri heif jbig jpeg2k lcms lzma raw svg tiff webp xml
External resources
- Free Software Foundation Europe - Infrastructure living the ideals of software freedom
- Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc) - Technology will (and does) degrade
- Keep a knowledge log
- ↑ https://docs.nextcloud.com/server/21/admin_manual/configuration_database/db_conversion.html
- ↑ https://docs.nextcloud.com/server/20/admin_manual/configuration_database/linux_database_configuration.html
- ↑ https://www.php.net/manual/en/set.mysqlinfo.php
- ↑ https://github.com/nextcloud/server/issues/26085