drastically improve the reliability of rbenv commands

README.md

@@ -15,7 +15,7 @@ The [wiki of this repo](https://github.com/jakejarvis/mastodon-utils/wiki) and t
 
 ***AGAIN, DEFINITELY DO NOT JUST RUN THIS IF YOU'RE NOT ME!!! 😊***
 
-This sets up the bare minimum customizations **after** Mastodon is installed:
+This sets up the bare minimum customizations ***after*** Mastodon is installed:
 
 ```sh
 git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils && cd /home/mastodon/utils
@@ -23,19 +23,42 @@ git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils
 # setup nginx using conf files from this repo:
 ./scripts/setup_nginx.sh
 
-# apply vanilla (and glitch-soc) patches from this repo:
+# back up Postgres, Redis, and secrets:
-./scripts/apply_patches.sh
+./scripts/backup.sh
+
+# pull latest Mastodon (vanilla or glitch-soc) and apply patches from this repo:
+./scripts/upgrade.sh
 
 # cherry-pick everything else below...
 ```
 
+## Scripts
+
+- [`init.sh`](init.sh): A small helper that runs at the very beginning of each script below to initialize `rbenv` and set consistent environment variables.
+  - **Optional:** To make your life easier, you can also source this script from the `.bashrc` of the `mastodon` user and/or whichever user you regularly SSH in as:
+
+```sh
+if [ -f /home/mastodon/utils/init.sh ]; then
+  . /home/mastodon/utils/init.sh
+fi
+```
+
+- [`apply_patches.sh`](scripts/apply_patches.sh): Dangerously applies ***every patch*** listed below, and automatically detects if `glitch-soc` patches should also be applied
+- [`backup.sh`](scripts/backup.sh): Backs up Postgres, Redis, and `.env.production` secrets to a `.tar.gz` file in `/home/mastodon/backups` — useful for a [periodic cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cron-jobs#backups)
+- [`setup_nginx.sh`](scripts/setup_nginx.sh): Sets up symlinks from `/etc/nginx` to nginx confs in this repo
+- [`upgrade.sh`](scripts/upgrade.sh): Upgrades Mastodon server (latest version if vanilla Mastodon, latest commit if `glitch-soc`) and re-applies patches listed below
+- [`version.sh`](scripts/version.sh): Tests `init.sh` by printing Mastodon, Ruby, and rbenv versions.
+- [`weekly_cleanup.sh`](scripts/weekly_cleanup.sh): Runs Mastodon's built-in [cleanup commands](https://docs.joinmastodon.org/admin/setup/#cleanup), designed for a [weekly cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cron-jobs#media-cleanup)
+  - Keeps 7 days of media (in object storage)
+  - Keeps 90 days of generated preview cards
+
 ## Patches
 
 #### Vanilla and `glitch-soc`:
 
-- [`favicons.patch`](patches/favicons.patch): Use custom icon images instead of Mastodon logo
 - [`robots.patch`](patches/robots.patch): Disallow search engines for all of Mastodon
 - [`increase-sidekiq-timeout.patch`](patches/increase-sidekiq-timeout.patch): Small bump in Sidekiq's timeout before it decides a remote instance isn't available. **Use this one very carefully!**
+- [`favicons.patch`](patches/favicons.patch): Use custom icon images instead of Mastodon logo
 - [`system-font.patch`](patches/system-font.patch): Use the system's default sans-serif font stack instead of Roboto
   - [`glitch/system-font.patch`](patches/glitch/system-font.patch)
 - [`hide-contact-email.patch`](patches/hide-contact-email.patch): Hides the `mailto:` link on the About page
@@ -50,20 +73,6 @@ git clone https://github.com/jakejarvis/mastodon-utils.git /home/mastodon/utils
   - [`remove-glitch-cruft.patch`](patches/glitch/remove-glitch-cruft.patch): Removes a bunch of junk no longer used by `glitch-soc`
   - [`sidebar-logo.patch`](patches/glitch/sidebar-logo.patch): Restore Mastodon logo in logged-out sidebar
 
-## Scripts
-
-- [`apply_patches.sh`](scripts/apply_patches.sh): Dangerously applies ***every patch*** listed above, and automatically detects if `glitch-soc` patches should also be applied
-- [`backup.sh`](scripts/backup.sh): Backs up Postgres, Redis, and `.env.production` secrets to a `.tar.gz` file in `/home/mastodon/backups` — useful for a periodic cronjob
-- [`setup_nginx.sh`](scripts/setup_nginx.sh): Sets up symlinks from `/etc/nginx` to nginx confs in this repo
-- [`tootctl_shim.sh`](scripts/tootctl_shim.sh): Small shell shim to run `tootctl` in `/home/mastodon/live` as the `mastodon` user by anyone & from anywhere
-  - Add this line to the `.bash_profile` of the user you normally login as:
-    - `. /home/mastodon/utils/scripts/tootctl_shim.sh`
-- [`upgrade.sh`](scripts/upgrade.sh): Upgrades Mastodon server (latest version if vanilla Mastodon, latest commit if `glitch-soc`) and re-applies patches listed above
-- [`version.sh`](scripts/version.sh): Tests `tootctl_shim.sh` by printing Mastodon version (`tootctl version`)
-- [`weekly_cleanup.sh`](scripts/weekly_cleanup.sh): Runs Mastodon's built-in [cleanup commands](https://docs.joinmastodon.org/admin/setup/#cleanup), designed for a [weekly cronjob](https://github.com/jakejarvis/mastodon-utils/wiki/Cleanup-tasks)
-  - Keeps 7 days of media (in object storage)
-  - Keeps 90 days of generated preview cards
-
 ## License
 
 MIT

init.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+# default paths
+export MASTODON_ROOT=/home/mastodon
+export UTILS_ROOT="$MASTODON_ROOT/utils"      # this repository
+export APP_ROOT="$MASTODON_ROOT/live"         # actual Mastodon files
+export BACKUPS_ROOT="$MASTODON_ROOT/backups"  # backups destination
+export LOGS_ROOT="$MASTODON_ROOT/logs"        # logs destintation
+export RBENV_ROOT="$MASTODON_ROOT/.rbenv"     # rbenv (w/ ruby-build plugin) directory
+
+# ---
+
+# initialize rbenv manually
+if [ -d "$RBENV_ROOT" ]; then
+  eval "$($RBENV_ROOT/bin/rbenv init -)"
+else
+  echo "⚠️ Didn't find rbenv at '$MASTODON_ROOT/.rbenv', double check the paths set in utils/init.sh..."
+fi
+
+# check for Mastodon in set location
+if [ ! -d "$APP_ROOT" ]; then
+  echo "⚠️ Didn't find Mastodon at '$APP_ROOT', double check the paths set in utils/init.sh..."
+fi
+
+# clone this repo if it doesn't exist in the proper location
+# if [ ! -d "$UTILS_ROOT" ]; then
+#   echo "⚠️ Can't find mastodon-utils in '$UTILS_DIR', cloning it for you..."
+#   sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT"
+# fi
+
+# ---
+
+# run a given command as the 'mastodon' user; e.g. `as_mastodon whoami`
+as_mastodon() {
+  # don't do unnecessary sudo'ing if we're already mastodon
+  if [ "$(whoami)" != "mastodon" ]; then
+    sudo -u mastodon env "PATH=$PATH" "$@"
+  else
+    "$@"
+  fi
+}
+
+# run /home/mastodon/live/bin/tootctl as 'mastodon' in '/home/mastodon/live'; e.g. `tootctl version`
+tootctl() {
+  ( cd "$APP_ROOT" && as_mastodon RAILS_ENV=production ruby "$APP_ROOT/bin/tootctl" "$@" )
+}

scripts/apply_patches.sh

@@ -3,40 +3,25 @@
 # exit when any step fails
 set -euo pipefail
 
-# default paths
+# initialize path
-MASTODON_ROOT=/home/mastodon
+source "$(dirname "$(realpath "$0")")"/../init.sh
-APP_ROOT="$MASTODON_ROOT/live"
-UTILS_ROOT="$MASTODON_ROOT/utils"
-RBENV_ROOT="$MASTODON_ROOT/.rbenv"
-
-# clone this repo if it doesn't exist in the proper location
-if [ ! -d "$UTILS_ROOT" ]
-then
-  sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT"
-
-  # fix permissions
-  sudo chown -R mastodon:mastodon "$UTILS_ROOT"
-  sudo git config --global --add safe.directory "$UTILS_ROOT"
-fi
 
 # apply custom patches
 cd "$APP_ROOT"
-sudo -u mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/*.patch
+as_mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/*.patch
-if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ];
+if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ]; then
-then
   # apply additional glitch-only patches:
-  sudo -u mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/glitch/*.patch
+  as_mastodon git apply --reject --allow-binary-replacement "$UTILS_ROOT"/patches/glitch/*.patch
 fi
 
 # update dependencies
 echo "Updating deps..."
-sudo -u mastodon "$RBENV_ROOT/shims/bundle" install --jobs "$(getconf _NPROCESSORS_ONLN)"
+as_mastodon bundle install --jobs "$(getconf _NPROCESSORS_ONLN)"
-sudo -u mastodon yarn install --pure-lockfile --network-timeout 100000
+as_mastodon yarn install --pure-lockfile --network-timeout 100000
 
 # compile new assets
 echo "Compiling new assets..."
-sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/bundle" exec rails assets:precompile
+as_mastodon RAILS_ENV=production bundle exec rails assets:precompile
-sudo chown -R mastodon:mastodon "$APP_ROOT"
 
 # restart frontend
 echo "Restarting mastodon-web..."

scripts/backup.sh

@@ -1,31 +1,34 @@
 #!/bin/bash
 
+# cronjob ran once per week at 3 AM on Monday; see https://crontab.guru/#0_3_*_*_1
+# syntax for crontab -e:
+#   0 3 * * 1  bash -c "/home/mastodon/utils/scripts/backup.sh >> /home/mastodon/logs/cron.log 2>&1"
+
 # exit when any step fails
 set -euo pipefail
 
-# default paths
+# initialize path
-MASTODON_ROOT=/home/mastodon
+source "$(dirname "$(realpath "$0")")"/../init.sh
-APP_ROOT="$MASTODON_ROOT/live"
-BACKUPS_ROOT="$MASTODON_ROOT/backups"
 
-if [ "$(systemctl is-active mastodon-web.service)" = "active" ]
+if [ "$(systemctl is-active mastodon-web.service)" = "active" ]; then
-then
   echo "⚠️  Mastodon is currently running."
   echo "We'll start the backup anyways, but if it's a critical one, stop all Mastodon"
   echo "services first with 'systemctl stop mastodon-*' and run this again."
   echo ""
 fi
 
-if [ ! -d "$BACKUPS_ROOT" ]
+if [ ! -d "$BACKUPS_ROOT" ]; then
-then
+  as_mastodon mkdir -p "$BACKUPS_ROOT"
-  sudo mkdir -p "$BACKUPS_ROOT"
-  sudo chown -R mastodon:mastodon "$BACKUPS_ROOT"
 fi
 
-TEMP_DIR=$(sudo -u mastodon mktemp -d)
+if [ ! -d "$LOGS_ROOT" ]; then
+  as_mastodon mkdir -p "$LOGS_ROOT"
+fi
+
+TEMP_DIR=$(as_mastodon mktemp -d)
 
 echo "Backing up Postgres..."
-sudo -u mastodon pg_dump -Fc mastodon_production -f "$TEMP_DIR/postgres.dump"
+as_mastodon pg_dump -Fc mastodon_production -f "$TEMP_DIR/postgres.dump"
 
 echo "Backing up Redis..."
 sudo cp /var/lib/redis/dump.rdb "$TEMP_DIR/redis.rdb"

scripts/setup_nginx.sh

@@ -4,20 +4,8 @@
 # exit when any step fails
 set -euo pipefail
 
-# default paths
+# initialize path
-MASTODON_ROOT=/home/mastodon
+source "$(dirname "$(realpath "$0")")"/../init.sh
-APP_ROOT="$MASTODON_ROOT/live"
-UTILS_ROOT="$MASTODON_ROOT/utils"
-
-# clone this repo if it doesn't exist in the proper location
-if [ ! -d "$UTILS_ROOT" ]
-then
-  sudo -u mastodon git clone https://github.com/jakejarvis/mastodon-utils.git "$UTILS_ROOT"
-
-  # fix permissions
-  sudo chown -R mastodon:mastodon "$UTILS_ROOT"
-  sudo git config --global --add safe.directory "$UTILS_ROOT"
-fi
 
 # setup nginx config
 sudo rm -rf /etc/nginx/sites-available

scripts/tootctl_shim.sh

@@ -1,9 +0,0 @@
-#!/bin/sh
-
-MASTODON_ROOT=/home/mastodon
-APP_ROOT="$MASTODON_ROOT/live"
-RBENV_ROOT="$MASTODON_ROOT/.rbenv"
-
-tootctl() {
-  ( cd "$APP_ROOT" && sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/ruby" "$APP_ROOT/bin/tootctl" "$@" )
-}

scripts/upgrade.sh

@@ -3,30 +3,21 @@
 # exit when any step fails
 set -euo pipefail
 
-# default paths
+# initialize path
-MASTODON_ROOT=/home/mastodon
+source "$(dirname "$(realpath "$0")")"/../init.sh
-APP_ROOT="$MASTODON_ROOT/live"
-UTILS_ROOT="$MASTODON_ROOT/utils"
-RBENV_ROOT="$MASTODON_ROOT/.rbenv"
-
-# check for existing installation
-if [ ! -d "$APP_ROOT" ]; then
-  echo "$APP_ROOT doesn't exist, are you sure Mastodon is installed?"
-  exit 255
-fi
 
 # pull latest mastodon source
 cd "$APP_ROOT"
-sudo -u mastodon git fetch --all
+as_mastodon git fetch --all
-sudo -u mastodon git stash push --message "pre-upgrade changes"
+as_mastodon git stash push --message "pre-upgrade changes"
 if [ -d "$APP_ROOT/app/javascript/flavours/glitch" ]; then
   # glitch-soc (uses latest commits)
   echo "Pulling latest glitch-soc commits..."
-  sudo -u mastodon git checkout glitch-soc/main
+  as_mastodon git checkout glitch-soc/main
 else
   # vanilla (uses latest release)
   echo "Pulling latest Mastodon release..."
-  sudo -u mastodon git checkout "$(sudo -u mastodon git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)"
+  as_mastodon git checkout "$(as_mastodon git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)"
 fi
 
 # pull & apply latest patches
@@ -34,17 +25,17 @@ fi
 
 # create blank custom.css (this overrides any CSS set in the admin panel, but if that's not being used, then
 # this quickly saves a request to the backend)
-sudo -u mastodon touch "$APP_ROOT/public/custom.css"
+as_mastodon touch "$APP_ROOT/public/custom.css"
 
 # set new ruby version
-RUBY_VERSION="$(sudo -u mastodon cat $APP_ROOT/.ruby-version)"
+RUBY_VERSION="$(as_mastodon cat "$APP_ROOT"/.ruby-version)"
-sudo -u mastodon RUBY_CONFIGURE_OPTS=--with-jemalloc "$RBENV_ROOT/bin/rbenv" install "$RUBY_VERSION" || true
+as_mastodon RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install "$RUBY_VERSION"
-sudo -u mastodon "$RBENV_ROOT/bin/rbenv" global "$RUBY_VERSION"
+as_mastodon rbenv global "$RUBY_VERSION"
 
 # run migrations:
 # https://docs.joinmastodon.org/admin/upgrading/
 echo "Running pre-deploy database migrations..."
-sudo -u mastodon SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production DB_PORT=5432 "$RBENV_ROOT/shims/bundle" exec rails db:migrate
+as_mastodon SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate
 
 # restart mastodon
 echo "Restarting services (round 1/2)..."
@@ -52,9 +43,9 @@ sudo systemctl restart mastodon-web mastodon-sidekiq mastodon-streaming
 
 # clear caches & run post-deployment db migration
 echo "Clearing cache..."
-sudo -u mastodon RAILS_ENV=production "$RBENV_ROOT/shims/ruby" "$APP_ROOT/bin/tootctl" cache clear
+as_mastodon RAILS_ENV=production ruby "$APP_ROOT/bin/tootctl" cache clear
 echo "Running post-deploy database migrations..."
-sudo -u mastodon RAILS_ENV=production DB_PORT=5432 "$RBENV_ROOT/shims/bundle" exec rails db:migrate
+as_mastodon RAILS_ENV=production DB_PORT=5432 bundle exec rails db:migrate
 
 # restart mastodon again
 echo "Restarting services (round 2/2)..."

scripts/version.sh

@@ -1,7 +1,11 @@
-#!/bin/sh
+#!/bin/bash
 
-set -e
+# exit when any step fails
+set -euo pipefail
 
-. "$(dirname "$0")/tootctl_shim.sh"
+# initialize path
+source "$(dirname "$(realpath "$0")")"/../init.sh
 
-tootctl version
+echo "* rbenv version: $(rbenv --version)"
+echo "* Ruby version: $(ruby --version)"
+echo "* Mastodon version: $(tootctl version)"

scripts/weekly_cleanup.sh

@@ -1,12 +1,18 @@
-#!/bin/sh
+#!/bin/bash
 
 # cronjob ran once per week at 3 AM on Sunday; see https://crontab.guru/#0_3_*_*_0
 # syntax for crontab -e:
-#   0 3 * * 0  root  /home/mastodon/utils/weekly_cleanup.sh >> /home/mastodon/logs/cron.log 2>&1
+#   0 3 * * 0  bash -c "/home/mastodon/utils/scripts/weekly_cleanup.sh >> /home/mastodon/logs/cron.log 2>&1"
 
-set -e
+# exit when any step fails
+set -euo pipefail
 
-. "$(dirname "$0")/tootctl_shim.sh"
+# initialize path
+source "$(dirname "$(realpath "$0")")"/../init.sh
+
+if [ ! -d "$LOGS_ROOT" ]; then
+  as_mastodon mkdir -p "$LOGS_ROOT"
+fi
 
 tootctl media remove --days 7
 tootctl preview_cards remove --days 90