diff --git a/README.md b/README.md index 07c4a64..55c083a 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ The [wiki of this repo](https://github.com/jakejarvis/mastodon-utils/wiki) and t ***AGAIN, DEFINITELY DO NOT JUST RUN THIS IF YOU'RE NOT ME!!! 😊*** -This sets up the bare minimum customizations **after** Mastodon is installed: +This sets up the bare minimum customizations ***after*** Mastodon is installed: ```sh git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils && cd /home/mastodon/utils @@ -23,19 +23,42 @@ git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils # setup nginx using conf files from this repo: ./scripts/setup_nginx.sh -# apply vanilla (and glitch-soc) patches from this repo: -./scripts/apply_patches.sh +# back up Postgres, Redis, and secrets: +./scripts/backup.sh + +# pull latest Mastodon (vanilla or glitch-soc) and apply patches from this repo: +./scripts/upgrade.sh # cherry-pick everything else below... ``` +## Scripts + +- [`init.sh`](init.sh): A small helper that runs at the very beginning of each script below to initialize `rbenv` and set consistent environment variables. + - **Optional:** To make your life easier, you can also source this script from the `.bashrc` of the `mastodon` user and/or whichever user you regularly SSH in as: + +```sh +if [ -f /home/mastodon/utils/init.sh ]; then + . /home/mastodon/utils/init.sh +fi +``` + +- [`apply_patches.sh`](scripts/apply_patches.sh): Dangerously applies ***every patch*** listed below, and automatically detects if `glitch-soc` patches should also be applied +- [`backup.sh`](scripts/backup.sh): Backs up Postgres, Redis, and `.env.production` secrets to a `.tar.gz` file in `/home/mastodon/backups` — useful for a [periodic cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cron-jobs#backups) +- [`setup_nginx.sh`](scripts/setup_nginx.sh): Sets up symlinks from `/etc/nginx` to nginx confs in this repo +- [`upgrade.sh`](scripts/upgrade.sh): Upgrades Mastodon server (latest version if vanilla Mastodon, latest commit if `glitch-soc`) and re-applies patches listed below +- [`version.sh`](scripts/version.sh): Tests `init.sh` by printing Mastodon, Ruby, and rbenv versions. +- [`weekly_cleanup.sh`](scripts/weekly_cleanup.sh): Runs Mastodon's built-in [cleanup commands](https://docs.joinmastodon.org/admin/setup/#cleanup), designed for a [weekly cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cron-jobs#media-cleanup) + - Keeps 7 days of media (in object storage) + - Keeps 90 days of generated preview cards + ## Patches #### Vanilla and `glitch-soc`: -- [`favicons.patch`](patches/favicons.patch): Use custom icon images instead of Mastodon logo - [`robots.patch`](patches/robots.patch): Disallow search engines for all of Mastodon - [`increase-sidekiq-timeout.patch`](patches/increase-sidekiq-timeout.patch): Small bump in Sidekiq's timeout before it decides a remote instance isn't available. **Use this one very carefully!** +- [`favicons.patch`](patches/favicons.patch): Use custom icon images instead of Mastodon logo - [`system-font.patch`](patches/system-font.patch): Use the system's default sans-serif font stack instead of Roboto - [`glitch/system-font.patch`](patches/glitch/system-font.patch) - [`hide-contact-email.patch`](patches/hide-contact-email.patch): Hides the `mailto:` link on the About page @@ -50,20 +73,6 @@ git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils - [`remove-glitch-cruft.patch`](patches/glitch/remove-glitch-cruft.patch): Removes a bunch of junk no longer used by `glitch-soc` - [`sidebar-logo.patch`](patches/glitch/sidebar-logo.patch): Restore Mastodon logo in logged-out sidebar -## Scripts - -- [`apply_patches.sh`](scripts/apply_patches.sh): Dangerously applies ***every patch*** listed above, and automatically detects if `glitch-soc` patches should also be applied -- [`backup.sh`](scripts/backup.sh): Backs up Postgres, Redis, and `.env.production` secrets to a `.tar.gz` file in `/home/mastodon/backups` — useful for a periodic cronjob -- [`setup_nginx.sh`](scripts/setup_nginx.sh): Sets up symlinks from `/etc/nginx` to nginx confs in this repo -- [`tootctl_shim.sh`](scripts/tootctl_shim.sh): Small shell shim to run `tootctl` in `/home/mastodon/live` as the `mastodon` user by anyone & from anywhere - - Add this line to the `.bash_profile` of the user you normally login as: - - `. /home/mastodon/utils/scripts/tootctl_shim.sh` -- [`upgrade.sh`](scripts/upgrade.sh): Upgrades Mastodon server (latest version if vanilla Mastodon, latest commit if `glitch-soc`) and re-applies patches listed above -- [`version.sh`](scripts/version.sh): Tests `tootctl_shim.sh` by printing Mastodon version (`tootctl version`) -- [`weekly_cleanup.sh`](scripts/weekly_cleanup.sh): Runs Mastodon's built-in [cleanup commands](https://docs.joinmastodon.org/admin/setup/#cleanup), designed for a [weekly cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cleanup-tasks) - - Keeps 7 days of media (in object storage) - - Keeps 90 days of generated preview cards - ## License MIT diff --git a/init.sh b/init.sh new file mode 100755 index 0000000..4cbff14 --- /dev/null +++ b/init.sh @@ -0,0 +1,46 @@ +#!/bin/bash + +# default paths +export MASTODON_ROOT=/home/mastodon +export UTILS_ROOT="$MASTODON_ROOT/utils" # this repository +export APP_ROOT="$MASTODON_ROOT/live" # actual Mastodon files +export BACKUPS_ROOT="$MASTODON_ROOT/backups" # backups destination +export LOGS_ROOT="$MASTODON_ROOT/logs" # logs destintation +export RBENV_ROOT="$MASTODON_ROOT/.rbenv" # rbenv (w/ ruby-build plugin) directory + +# --- + +# initialize rbenv manually +if [ -d "$RBENV_ROOT" ]; then + eval "$($RBENV_ROOT/bin/rbenv init -)" +else + echo "⚠️ Didn't find rbenv at '$MASTODON_ROOT/.rbenv', double check the paths set in utils/init.sh..." +fi + +# check for Mastodon in set location +if [ ! -d "$APP_ROOT" ]; then + echo "⚠️ Didn't find Mastodon at '$APP_ROOT', double check the paths set in utils/init.sh..." +fi + +# clone this repo if it doesn't exist in the proper location +# if [ ! -d "$UTILS_ROOT" ]; then +# echo "⚠️ Can't find mastodon-utils in '$UTILS_DIR', cloning it for you..." +# sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT" +# fi + +# --- + +# run a given command as the 'mastodon' user; e.g. `as_mastodon whoami` +as_mastodon() { + # don't do unnecessary sudo'ing if we're already mastodon + if [ "$(whoami)" != "mastodon" ]; then + sudo -u mastodon env "PATH=$PATH" "$@" + else + "$@" + fi +} + +# run /home/mastodon/live/bin/tootctl as 'mastodon' in '/home/mastodon/live'; e.g. `tootctl version` +tootctl() { + ( cd "$APP_ROOT" && as_mastodon RAILS_ENV=production ruby "$APP_ROOT/bin/tootctl" "$@" ) +} diff --git a/scripts/apply_patches.sh b/scripts/apply_patches.sh index 3f630aa..5845079 100755 --- a/scripts/apply_patches.sh +++ b/scripts/apply_patches.sh @@ -3,40 +3,25 @@ # exit when any step fails set -euo pipefail -# default paths -MASTODON_ROOT=/home/mastodon -APP_ROOT="$MASTODON_ROOT/live" -UTILS_ROOT="$MASTODON_ROOT/utils" -RBENV_ROOT="$MASTODON_ROOT/.rbenv" - -# clone this repo if it doesn't exist in the proper location -if [ ! -d "$UTILS_ROOT" ] -then - sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT" - - # fix permissions - sudo chown -R mastodon:mastodon "$UTILS_ROOT" - sudo git config --global --add safe.directory "$UTILS_ROOT" -fi +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh # apply custom patches cd "$APP_ROOT" -sudo -u mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/*.patch -if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ]; -then +as_mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/*.patch +if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ]; then # apply additional glitch-only patches: - sudo -u mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/glitch/*.patch + as_mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/glitch/*.patch fi # update dependencies echo "Updating deps..." -sudo -u mastodon "$RBENV_ROOT/shims/bundle" install --jobs "$(getconf _NPROCESSORS_ONLN)" -sudo -u mastodon yarn install --pure-lockfile --network-timeout 100000 +as_mastodon bundle install --jobs "$(getconf _NPROCESSORS_ONLN)" +as_mastodon yarn install --pure-lockfile --network-timeout 100000 # compile new assets echo "Compiling new assets..." -sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/bundle" exec rails assets:precompile -sudo chown -R mastodon:mastodon "$APP_ROOT" +as_mastodon RAILS_ENV=production bundle exec rails assets:precompile # restart frontend echo "Restarting mastodon-web..." diff --git a/scripts/backup.sh b/scripts/backup.sh index 6670777..458a0e3 100755 --- a/scripts/backup.sh +++ b/scripts/backup.sh @@ -1,31 +1,34 @@ #!/bin/bash +# cronjob ran once per week at 3 AM on Monday; see https://crontab.guru/#0_3_*_*_1 +# syntax for crontab -e: +# 0 3 * * 1 bash -c "/home/mastodon/utils/scripts/backup.sh >> /home/mastodon/logs/cron.log 2>&1" + # exit when any step fails set -euo pipefail -# default paths -MASTODON_ROOT=/home/mastodon -APP_ROOT="$MASTODON_ROOT/live" -BACKUPS_ROOT="$MASTODON_ROOT/backups" +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh -if [ "$(systemctl is-active mastodon-web.service)" = "active" ] -then +if [ "$(systemctl is-active mastodon-web.service)" = "active" ]; then echo "⚠️ Mastodon is currently running." echo "We'll start the backup anyways, but if it's a critical one, stop all Mastodon" echo "services first with 'systemctl stop mastodon-*' and run this again." echo "" fi -if [ ! -d "$BACKUPS_ROOT" ] -then - sudo mkdir -p "$BACKUPS_ROOT" - sudo chown -R mastodon:mastodon "$BACKUPS_ROOT" +if [ ! -d "$BACKUPS_ROOT" ]; then + as_mastodon mkdir -p "$BACKUPS_ROOT" fi -TEMP_DIR=$(sudo -u mastodon mktemp -d) +if [ ! -d "$LOGS_ROOT" ]; then + as_mastodon mkdir -p "$LOGS_ROOT" +fi + +TEMP_DIR=$(as_mastodon mktemp -d) echo "Backing up Postgres..." -sudo -u mastodon pg_dump -Fc mastodon_production -f "$TEMP_DIR/postgres.dump" +as_mastodon pg_dump -Fc mastodon_production -f "$TEMP_DIR/postgres.dump" echo "Backing up Redis..." sudo cp /var/lib/redis/dump.rdb "$TEMP_DIR/redis.rdb" diff --git a/scripts/setup_nginx.sh b/scripts/setup_nginx.sh index 7c10bdc..5cc4825 100755 --- a/scripts/setup_nginx.sh +++ b/scripts/setup_nginx.sh @@ -4,20 +4,8 @@ # exit when any step fails set -euo pipefail -# default paths -MASTODON_ROOT=/home/mastodon -APP_ROOT="$MASTODON_ROOT/live" -UTILS_ROOT="$MASTODON_ROOT/utils" - -# clone this repo if it doesn't exist in the proper location -if [ ! -d "$UTILS_ROOT" ] -then - sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT" - - # fix permissions - sudo chown -R mastodon:mastodon "$UTILS_ROOT" - sudo git config --global --add safe.directory "$UTILS_ROOT" -fi +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh # setup nginx config sudo rm -rf /etc/nginx/sites-available diff --git a/scripts/tootctl_shim.sh b/scripts/tootctl_shim.sh deleted file mode 100755 index 94f246b..0000000 --- a/scripts/tootctl_shim.sh +++ /dev/null @@ -1,9 +0,0 @@ -#!/bin/sh - -MASTODON_ROOT=/home/mastodon -APP_ROOT="$MASTODON_ROOT/live" -RBENV_ROOT="$MASTODON_ROOT/.rbenv" - -tootctl() { - ( cd "$APP_ROOT" && sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/ruby" "$APP_ROOT/bin/tootctl" "$@" ) -} diff --git a/scripts/upgrade.sh b/scripts/upgrade.sh index 3989e7e..875b2d2 100755 --- a/scripts/upgrade.sh +++ b/scripts/upgrade.sh @@ -3,30 +3,21 @@ # exit when any step fails set -euo pipefail -# default paths -MASTODON_ROOT=/home/mastodon -APP_ROOT="$MASTODON_ROOT/live" -UTILS_ROOT="$MASTODON_ROOT/utils" -RBENV_ROOT="$MASTODON_ROOT/.rbenv" - -# check for existing installation -if [ ! -d "$APP_ROOT" ]; then - echo "$APP_ROOT doesn't exist, are you sure Mastodon is installed?" - exit 255 -fi +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh # pull latest mastodon source cd "$APP_ROOT" -sudo -u mastodon git fetch --all -sudo -u mastodon git stash push --message "pre-upgrade changes" +as_mastodon git fetch --all +as_mastodon git stash push --message "pre-upgrade changes" if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ]; then # glitch-soc (uses latest commits) echo "Pulling latest glitch-soc commits..." - sudo -u mastodon git checkout glitch-soc/main + as_mastodon git checkout glitch-soc/main else # vanilla (uses latest release) echo "Pulling latest Mastodon release..." - sudo -u mastodon git checkout "$(sudo -u mastodon git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)" + as_mastodon git checkout "$(as_mastodon git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)" fi # pull & apply latest patches @@ -34,17 +25,17 @@ fi # create blank custom.css (this overrides any CSS set in the admin panel, but if that's not being used, then # this quickly saves a request to the backend) -sudo -u mastodon touch "$APP_ROOT/public/custom.css" +as_mastodon touch "$APP_ROOT/public/custom.css" # set new ruby version -RUBY_VERSION="$(sudo -u mastodon cat $APP_ROOT/.ruby-version)" -sudo -u mastodon RUBY_CONFIGURE_OPTS=--with-jemalloc "$RBENV_ROOT/bin/rbenv" install "$RUBY_VERSION" || true -sudo -u mastodon "$RBENV_ROOT/bin/rbenv" global "$RUBY_VERSION" +RUBY_VERSION="$(as_mastodon cat "$APP_ROOT"/.ruby-version)" +as_mastodon RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install "$RUBY_VERSION" +as_mastodon rbenv global "$RUBY_VERSION" # run migrations: # https://docs.joinmastodon.org/admin/upgrading/ echo "Running pre-deploy database migrations..." -sudo -u mastodon SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production DB_PORT=5432 "$RBENV_ROOT/shims/bundle" exec rails db:migrate +as_mastodon SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate # restart mastodon echo "Restarting services (round 1/2)..." @@ -52,9 +43,9 @@ sudo systemctl restart mastodon-web mastodon-sidekiq mastodon-streaming # clear caches & run post-deployment db migration echo "Clearing cache..." -sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/ruby" "$APP_ROOT/bin/tootctl" cache clear +as_mastodon RAILS_ENV=production ruby "$APP_ROOT/bin/tootctl" cache clear echo "Running post-deploy database migrations..." -sudo -u mastodon RAILS_ENV=production DB_PORT=5432 "$RBENV_ROOT/shims/bundle" exec rails db:migrate +as_mastodon RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate # restart mastodon again echo "Restarting services (round 2/2)..." diff --git a/scripts/version.sh b/scripts/version.sh index 1ecdaa6..2073715 100755 --- a/scripts/version.sh +++ b/scripts/version.sh @@ -1,7 +1,11 @@ -#!/bin/sh +#!/bin/bash -set -e +# exit when any step fails +set -euo pipefail -. "$(dirname "$0")/tootctl_shim.sh" +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh -tootctl version +echo "* rbenv version: $(rbenv --version)" +echo "* Ruby version: $(ruby --version)" +echo "* Mastodon version: $(tootctl version)" diff --git a/scripts/weekly_cleanup.sh b/scripts/weekly_cleanup.sh index 5cd0396..4a459d0 100755 --- a/scripts/weekly_cleanup.sh +++ b/scripts/weekly_cleanup.sh @@ -1,12 +1,18 @@ -#!/bin/sh +#!/bin/bash # cronjob ran once per week at 3 AM on Sunday; see https://crontab.guru/#0_3_*_*_0 # syntax for crontab -e: -# 0 3 * * 0 root /home/mastodon/utils/weekly_cleanup.sh >> /home/mastodon/logs/cron.log 2>&1 +# 0 3 * * 0 bash -c "/home/mastodon/utils/scripts/weekly_cleanup.sh >> /home/mastodon/logs/cron.log 2>&1" -set -e +# exit when any step fails +set -euo pipefail -. "$(dirname "$0")/tootctl_shim.sh" +# initialize path +source "$(dirname "$(realpath "$0")")"/../init.sh + +if [ ! -d "$LOGS_ROOT" ]; then + as_mastodon mkdir -p "$LOGS_ROOT" +fi tootctl media remove --days 7 tootctl preview_cards remove --days 90