Deploying Rails on OpenBSD
Table of content:
- Why OpenBSD?
- Setting up the server at Vultr.com
- First login and base setup
- Install and setup needed packages
- Firewall setup
- Setup email to get sysadmin emails from the operating system
- Setup the HTTP web server, proxy, and certificates for TLS
- Rails project setup
- Backups
- Deployment workflow
- Keeping the server up to date
- Optional extra step: Add an IMAP service so you can view sysadmin emails directly from your devices
Do you have any comments or thoughts about this guide? Find any mistakes? Please email me at et@erict.org.
TL;DR: I deployed my Rails 7.1 app on an OpenBSD 7.5 server. OpenBSD as an OS that focuses on simplicity and security. It worked fine with my Rails app, but took some effort to set up. If you want low-friction deployment you should probably just use Linux for Rails apps. Recent versions of Rails also comes with a Dockerfile (which is linux native) for deployment, a template for GitHub Action CI workflow (which does not have OpenBSD runners), and is preconfigured with Kamal 2. All of these are made with linux in mind.
Why use OpenBSD for Rails applications?
Here are the main reasons for me:
- One of their goals is to “Try to be the #1 most secure operating system”. They have a great track record and regularly do security audits to search for and fix new security holes.
- It is simple and the software is reliable. They are careful about what they let into the OS.
- I like their project goals.
I use it for the server running erict.org, my email server, and my own encrypted cloud storage. For these purposes I am very happy with OpenBSD and I have no plan on changing OS.
Since I was so happy with it for my personal server, I wanted to see how it was to run Rails on it.
Why might you not want to use OpenBDS for Rails apps?
The later versions of Rails comes with default configurations and tools that are meant to be running on Linux: The Dockerfile and Dev container, GitHub Actions config, and Kamal deployment. You don’t have to use them, but they do simplify deployment (and slightly also development through dev containers). How they relate to OpenBSD:
- Docker requires linux or a linux VM. On OpenBSD you need to run a linux VM (manually) in order to use Docker.
- Dev containers are used to run your Rails application in a container (Docker), without needing to install Ruby or Rails or its dependencies directly on your machine.
- With GitHub Actions you can choose to run jobs and workflows (e.g. automated tests) on Linux, Windows or MacOS machines. BSD-based operating systems are not in the list.
- Kamal simplifies deployment. Especially when deploying to multiple machines. It uses Docker containers.
My thoughts
The Rails team is certainly targeting the Linux platform, and it will be easier to deploy on that platform because of the default configs and tools.
I will personally go with Linux if I make other Rails apps.
But if you want OpenBSD, here is the guide.. :)
Setting up the server at Vultr.com
I used Vultr.com because I have used them before and never had any problems. But this guide should work with any company that provides servers with OpenBSD.
-
Go to Vultr.com. Make an account and log in, then click the “Deploy +” button in the top right corner to deploy a new instance.
-
Choose instance settings. I chose this $6/month setup:
Type: Cloud compute - shared CPU.
Location: Choose the one closest to you.
Image: OpenBSD 7.5.
Plan: 25 GB NVMe.
Disable auto backups (I do backups myself).
Add your SSH key for login.
Choose a hostname.
-
Click “Deploy Now” and wait a bit for the instance to start up.
First login and base setup
Login as the root user
Find your server’s IP on your Vultr.com dashboard.
SSH login should work without password if you added your SSH public key when deploying the instance.
ssh root@YOUR.IP
Update the system
root$ syspatch
Add new user
Choose defaults unless you know otherwise. Enter username and password when prompted.
root$ adduser
Deny root logins and login by password
modify the sshd_config
:
root$ vi /etc/ssh/sshd_config
# /etc/ssh/sshd_config
...
PermitRootLogin no
...
PasswordAuthentication no
...
Enable SSH login for your new user
Copy the authorized_keys file from the root user to the new user. The root user got this entry from the Vultr.com instance setup.
root$ cp /root/.ssh/authorized_keys /home/NEW_USER/.ssh/authorized_keys
Enable doas for running commands as root
Doas is similar to sudo, but doas is simpler and was created to have a smaller attack surface and being easier to audit.
root$ cp /etc/examples/doas.conf /etc/doas.conf
root$ vi /etc/doas.conf
Add the following two lines. This enables doas without password for both your new user and the root user.
# /etc/doas.conf
...
permit nopass NEW_USER
permit nopass root
Optionally disable IPv6 and sound
root$ rcctl disable slaacd sndiod
Reboot and login as your new user
root$ exit
$ ssh NEW_USER@YOUR_SERVER_IP
Install and setup needed packages
If you are logged in as your new user, you have to prepend all package
installation commands with doas
.
Ruby
Install the correct version for your Rails app.
$ pkg_info -Q ruby # See available versions
$ doas pkg_add ruby-3.2.4 # Install the version you want
Nokogiri
Nokogiri is a dependency for Rails. “Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.”
$ pg_add libiconv gtar
$ alias tar=gtar
$ export GEM_HOME="$HOME/.gem"
$ gem install nokogiri
See the website for more information.
Postgresql
Postgresql is by default accessed on 127.0.0.1:5432
.
$ pkg_info -Q postgres-server # see versions available
$ pkg_add postgres-server-16.4 # install whatever version you want
Now we have to initialize the database as the
_postgresql
user:
$ doas -u _postgresql initdb -D /var/postgresql/data/ -U postgres
Enable auto-start of postgresql on boot, then start it:
$ doas rcctl enable postgresql
$ doas rcctl start postgresql
Enter psql
and create a role
for the rails
app::
$ doas -u _postgresql psql -U postgres
postgres=/ CREATE ROLE myapp LOGIN SUPERUSER PASSWORD 'your-secret-password';
Later on this will be added to Rails encrypted credentials file.
Redis
Used for ActionCable. Accessed on
redis://localhost:6379/1
.
$ pkg_add redis
Enable start on boot, and start redis:
$ rcctl enable redis
$ rcctol start redis
Git
First install Git.
pkg_add git
Then add a “deploy SSH key” to Github, so your Vultr server can pull from the Github repo.
Read instructions here.
Then verify by cloning your Rails app to the home directory of the new user:
$ cd /home/NEW_USER/
$ git pull YOUR_GITHUB_REPO
Rails
Install rails:
$ gem install rails
If you get the warning that you dont have the ruby binary in your
PATH
, add this to your .profile
:
# ~/.profile
...
# Change to the correct location and version for your ruby binary
PATH="$HOME/.local/share/gem/ruby/3.3/bin:$PATH"
Move into the project directory (which you got from git clone in previous step), and run bundle install:
$ cd RAILS_PROJECT
$ bundle install
If you get the error
Bundler::PermissionError: There was an error while trying to write
to `/usr/local/lib/ruby/gems/3.3/cache/…
, then change the location for where you install your gems to a place
where your user has permission:
$ bundle config set path /home/NEW_USER/.bundle/
ActiveStorage dependencies
Install vips for image editing.
$ pkg_add libvips
Mutt
Mutt is a terminal-based email client. This is used to view system administration emails from the OS.
$ pkg_add curl mutt--sasl
Firewall setup
pf is the firewall (packet filter). Add a pf configuration file, if it is not already there.
$ touch /etc/pf.conf
Then add the following configuration to /etc/pf.conf
.
# /etc/pf.conf
# The interface is called vio0. This may be different on non-vulture servers.
if="vio0"
set skip on lo
block drop all
pass out on $if
pass quick proto tcp from any to $if port { 22, 25, 80, 443, 587, 993 } flags S/SA keep state
Remove the ports you don’t need:
- 22: SSH
- 25: SMTP
- 80: HTTP
- 443: HTTPS
- 587: SMTP
- 993: IMAP
Setup email to get sysadmin emails from the operating system
This step enables you to view system messages locally with the Mutt email client. Mutt is a nice little text-based email client, good for managing these kinds of emails. OpenBSD comes with OpenSMTPD, which we will use to handle emails for us.
Create the Maildir folder for holding your emails, and a
.muttrc
file for your Mutt configuration.
$ cd ~
$ mkdir -p Maildir/{cur,new,tmp}
$ chmod -R 700 Maildir
$ touch .muttrc
$ vi .muttrc
Copy the following configuration into that file:
# ~/.muttrc
set mbox_type=maildir
set spoolfile="/home/NEW_USER/Maildir/"
set folder="/home/NEW_USER/Maildir/"
auto_view text/html
alternative_order text/plain text/html
Add your new username to the aliases
file:
$ doas vi /etc/mail/aliases
# /etc/mail/aliases
# Line 85. Add your new user as an alias to root, manager, and dumper
...
root: NEW_USER
manager: NEW_USER
dumper: NEW_USER
Make small change to smtpd.conf
:
$ doas vi /etc/mail/smtpd.conf
# /etc/mail/smtpd.conf
# Go to line 14. Change mbox to maildir
...
action "local_mail" maildir alias <aliases>
...
Restart the email service:
$ doas rcctl restart smtpd
Now you can view emails from the system with Mutt. These are things like cron job errors, results from security script that the OS runs, and so on.
$ mutt
Setup the HTTP web server, proxy, and certificates for TLS
httpd
httpd is an HTTP server that comes with OpenBSD.
Copy the following configuration into /etc/httpd.conf
,
changing "EXAMPLE.COM"
to your own domain name.
The main purpose for httpd for this setup is to listen to incoming acme-challenge requests, which is needed to create our TLS/SSL certificate.
# /etc/httpd.conf
prefork 5
types {
include "/usr/share/misc/mime.types"
}
server "EXAMPLE.COM" {
listen on * port 80
location "/.well-known/acme-challenge/*" {
root "/acme"
request strip 2 # strip the first two paths of the url (.well-known/acme-challenge/)
}
location * {
block return 301 "https://$HTTP_HOST$REQUEST_URI"
}
}
Enable and start the service.
$ rcctl enable httpd
$ rcctl start httpd
Relayd
Relayd is a daemon to relay and dynamically redirect incoming connections to a target host. Its main purposes are to run as a load-balancer, application layer gateway, or transparent proxy. It also gives control of the HTTP headers. Relayd with httpd is used instead of something like nginx.
For our purposes, relayd will listen to requests to port
443
, access your TLS keys, then modify the HTTP headers,
before lastly forwarding the request to your Rails app on
127.0.0.1:5000
.
We will create the SSL/TLS certificate and key in the next step.
Create an empty config file:
$ touch /etc/relayd.conf
Put the following config into that file, changing
“EXAMPLE.COM”
to your own domain. Read more about how how
the config works
here.
# /etc/relayd.conf
log state changes
log connection errors
prefork 5
table <localhost> { 127.0.0.1 }
# Protocols are templates defining settings and rules for relays.
http protocol «my_https" {
tls keypair "EXAMPLE.COM" # will look keypair in /etc/ssl
return error
match request header set "X-Forwarded-For" value "$REMOTE_ADDR"
match request header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"
# test in https://securityheaders.com
match response header remove "Server"
match response header append "Strict-Transport-Security" value "max-age=31536000"
match response header append "X-Frame-Options" value "SAMEORIGIN"
match response header append "X-XSS-Protection" value "1; mode=block"
match response header append "X-Content-Type-Options" value "nosniff"
match response header append "Referrer-Policy" value "strict-origin"
match response header append "Content-Security-Policy" value "default-src https: 'unsafe-inline'"
match response header append "Permissions-Policy" value "accelerometer=(none), camera=(none), geolocation=(none), gyroscope=(none), magnetometer=(none), microphone=(none), payment=(none), usb=(none)"
pass request quick header "Host" value "EXAMPLE.COM" forward to <localhost>
}
relay "https" {
listen on 0.0.0.0 port 443 tls
protocol «my_https» # Requests will go through the my_https sequence
forward to <localhost> port 5000
}
Enable relayd on startup, then start it.
$ rcctl enable relayd
$ rcctl start relayd
Note: If your Rails application is behind a load balancer or other proxy (like relayd or nginx), and that proxy is responsible for providing SSL/TLS (like we are doing here), then that proxy, not your Rails app, is responsible for ensuring HTTP requests are redirected to HTTPS, and your Rails app shouldn’t have to think about SSL/TLS at all.
In that case, the Rails config option
config.force_ssl
should be false, even in production.
Setup acme-client for managing TLS/SSL certificates
acme-client is an Automatic Certificate Management Environment (ACME) client. It looks in its configuration for a domain section and uses that configuration to retrieve an X.509 certificate which can be used to provide domain name validation (i.e. prove that the domain is who it says it is).
Create the configuration file:
$ touch /etc/acme-client.conf
Add the following configuration (see
here for an
explanation), changing "EXAMPLE.COM"
to your domain name:
# /etc/acme-client.conf
authority letsencrypt {
api url "https://acme-v02.api.letsencrypt.org/directory"
account key "/etc/acme/letsencrypt-privkey.pem"
}
authority letsencrypt-staging {
api url "https://acme-staging-v02.api.letsencrypt.org/directory"
account key "/etc/acme/letsencrypt-staging-privkey.pem"
}
domain EXAMPLE.COM {
alternative names { www.EXAMPLE.COM }
domain key "/etc/ssl/private/EXAMPLE.COM:443.key"
domain full chain certificate "/etc/ssl/EXAMPLE.COM:443.crt"
sign with letsencrypt
}
Now create a certificate by running:
$ acme-client EXAMPLE.COM
# it will generate the certificate to /etc/ssl/ (public) and /etc/ssl/private/ (private)
Whenever you renew your certificate, you need to restart the services that uses it. So restart relayd:
$ rcctl restart relayd
Now lets automate the certificate renewal by adding a cron job:
$ doas crontab -e
# crontab
...
# add the following line:
11 3 * * 5 acme-client EXAMPLE.COM && rcctl restart relayd
...
Optional: Adding subdomains for your certificate
For each subdomain, you need to do the following:
-
Add the subdomain (e.g.,
sub.domain.com
) to the alternative names section inacme-client.conf
.
Then run acme-client to renew the certificate.
$ doas acme-client EXAMPLE.COM
And reload relayd.
$ doas rcctl reload relayd
Starting everything
If you haven’t started the services yet, run these commands:
# first make sure we have certificates for TLS
acme-client -v your-domain.com
# then start everything
rcctl enable postgres
rcctl start postgres
rcctl enable httpd
rcctl start httpd
rcctl enable redis
rcctl start redis
rcctl enable relayd
rcctl start relayd
Rails project setup
First install missing gem executables with:
$ cd ~/YOUR_RAILS_PROJECT
$ bin/bundle install
If you use TailwindCSS
I used TailwindCSS for my app, but found out that Tailwind does not
have a binary that supports the x86_64-openbsd
platform.
So I had to include the tailwind.css
file that was built
during development to my production server.
I just included it into the Git repo.
The credentials file
The credentials file will hold our secrets for the database and AWS.
To edit the credentials file, run
EDOTIR=vi bin/rails credentials:edit
. This command will
create the credentials file if it does not exist. Additionally, this
command will create config/master.key
if no master key is
defined.
If you create a credential files locally during development, and want to use this same file in production, you have to somehow get the master key to the production server. Two ways to do this:
-
Copy over the
master.key
to the production server. -
Use the environment variable
ENV["RAILS_MASTER_KEY"]
to the master key value.
Setting up the database
We need to add the database username and password to the credentials file.
$ cd RAILS_PROJECT
$ EDITOR=vi bin/rails credentials:edit
Add the following:
# credentials
...
database:
username: <your database username> # we chose "myapp" when we setup the postgresql
password: <your database password>
Then make sure the config/database.yml
configuration
looks to the credentials file for the database authentication details:
$ vi config/database.yml
...
production:
adapter: postgresql
encoding: unicode
database: yourapp_production
pool: 5
username: <%= Rails.application.credentials.dig(:database, :username) %>
password: <%= Rails.application.credentials.dig(:database, :password) %>
Now either run migrations or load the schema (I loaded the schema):
$ bundle exec rake db:schema:load RAILS_ENV="production" DISABLE_DATABASE_ENVIRONMENT_CHECK=1
Seed the database:
$ bin/rails db:seed RAILS_ENV="production"
Optionally go into the console to try some queries to see that everything is OK:
$ bin/rails c -e production
Now I got the following error:
PG::InvalidParameterValue: ERROR: new encoding (UTF8) is
incompatible with the encoding of the template database
(SQL_ASCII)
Which I fixed by
adding the following to database.yml
:
# config/database.yml
....
production:
template: template0
Redis
Add the following to the action cable configuration:
# config/cable.yml
...
production:
adapter: redis
url: redis://localhost:6379/1
channel_prefix: APPNAME_production
ActiveStorage with AWS S3
I use ActiveStorage with AWS S3 for storing uploaded images.
First set the config/storage.yml
config to use AWS S3:
# config/storage.yml
...
amazon:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
region: eu-north-1 # YOUR REGION
bucket: taulag # YOUR BUCKET
public: true # May be app specific, but for me all images are public.
Add your AWS secrets to the credentials file:
# credentials
...
aws:
access_key_id: <your aws access key id>
secret_access_key: <your aws access key>
At this point I got the following errors when trying to upload a picture:
-
Could not open library 'libglib-2.0.so.0': File not found #221
-
LoadError (Could not open library 'gobject-2.0.so.0': File not found
-
(and same thing with libvips, but I don't have the error text)
But I checked and the libraries were already installed, so what I did was to create symbolic links to the installed libraries, with the names that got the error:
$ doas ln -s /usr/local/lib/libglib-2.0.so.4201.11 /usr/local/lib/libglib-2.0.so.0
$ doas ln -s /usr/local/lib/libgobject-2.0.so.4200.18 /usr/local/lib/libgobject-2.0.so.0
$ doas ln -s /usr/local/lib/libvips.so.1.0 /usr/local/lib/libvips.so.42
And it worked after that.
Modifying Puma config for port 5000
Puma is the ruby web server that binds the relayd proxy to our rails app.
In the relayd config we relayed traffic for our app to
127.0.0.1:5000
. We now need Puma to listen to this
address. So make this modification to the Puma config:
# config/puma.rb
...
bind 'tcp://127.0.0.1:5000'
...
Using the rails server command to veryfy that everything is working
When first starting the rails app, its nice to see that everything
works as expected. Using the bin/rails server
command
with --log-to-stdout
is handy for this:
bin/rails server -b 127.0.0.1 -p 5000 -e production --log-to-stdout
When everything works fine we will create a service (rc.d) for running and managing the app.
Using rc.d for managing the app
We can create a daemon service for the app using
rc.d. Once its created we
can start, stop, restart and status the app with:
rcctl start/stop/restart/status myappname
.
The /etc/rc.d
directory contains various scripts to
start, stop, and reconfigure daemon programs (“services”). You can
find services like postgres
and redis
here.
First create a file in /etc/rc.d
with your app name, and
add the following script, changing NEW_USER
to your new
OpenBSD user, and YOUR_RAILS_APP
to the directory of your
rails app. Read
rc.subr for mor info
about how this script works.
#!/bin/ksh
# /etc/rc.d/myappname
daemon="/home/NEW_USER/YOUR_RAILS_APP/bin/rails"
daemon_flags="s -b 127.0.0.1 -p 5000 -e production -d"
daemon_user="NEW_USER"
# Run in background
rc_bg=YES
. /etc/rc.d/rc.subr
rc_check() {
cd /home/eric/taulag
bundle exec pumactl status
}
rc_restart() {
cd /home/eric/taulag
bundle exec pumactl phased-restart
}
rc_stop() {
cd /home/eric/taulag
bundle exec pumactl stop
}
rc_cmd "$1"
Now make the script executable:
$ chmod +x /etc/rc.d/myappname
And enable automatic start on boot, then start the app:
$ doas rcctl enable myappname
$ doas rcctl start myappname
Your app should be running. Browse to your servers IP (or domain name) to verify.
Backups
To backup your postgresql database, we can add a cron job to dump the database:
$ doas crontab -e
# the -n option means that no mail is sent after a successful run. You will only get an email on error.
...
30 1 * * * -n pg_dump -U et PRODUCTION_DB_NAME -Fc > /home/NEW_USER/backup/your_db_backup.dump
I suggest also adding a cron job for sending this backup to another server. I send my backups to a Hetzner storage box:
30 1 * * * -n rsync -e 'ssh -p23' -avz /home/NEW_USER/backup HETZNER_USER@HETZNER_USER.your-storagebox.de:
In addition, I like to backup all configuration files (relayd, httpd, acme-client, etc.) by having a cron job that copies them to the backup folder.
Deployment workflow
Deploying is simple:
- Develop and test locally
git push
to githubssh
to production (OpenBSD) machinecd
into my rails projectgit pull
from githubdoas rcctl restart myappname
This works great for projects where I am the only developer, and I don’t feel the need to add a more complicated CI/CD pipeline. This deployment takes maybe 30 seconds.
Keeping the server up to date
You can just let the server run as is for a long time. But every six months a new release of OpenBSD is released, and if you want to update your server you can do the following:
SSH
into your serverdoas su
syspatch
fw_update
-
pkg_add -u
sysupgrade
exit; exit
to log out.
If the sysupgrade
step updated your operating system and
your server rebooted, then there is one more step:
SSH
into your serverdoas su
sysmerge
- Follow any instructions.
-
Re-do the
syspatch ; fw_update ; pkg_add -u
steps, above. - Type
exit; exit
to log out.
Optional extra step: Add an IMAP service so you can view sysadmin emails directly from your devices
First install dovecot, the service that will run IMAP (it will also run POP3):
$ doas pkg_add dovecot
# choose latest version, for me this was dovecot-2.3.21.1v0
# do not choose the -gssapi version
Add the below configuration, changing "EXAMPLE.COM"
to
your domain name.
$ doas vi /etc/dovecot/dovecot.conf
# /etc/dovecot/dovecot.conf Dovecot configuration file
# Protocols we want to be serving.
protocols = imap pop3
ssl_cert = </etc/ssl/EXAMPLE.COM:443.crt
ssl_key = </etc/ssl/private/EXAMPLE.COM:443.key
ssl = required
mail_location = maildir:~/Maildir
service imap-login {
# disable non-ssl-port
inet_listener imap {
port = 0
}
inet_listener imaps {
port = 993
}
}
service pop3-login {
# disable non-ssl-port
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
}
}
# PAM-like authentication for OpenBSD.
# <doc/wiki/PasswordDatabase.BSDAuth.txt>
passdb {
driver = bsdauth
}
userdb {
# <doc/wiki/AuthDatabase.Passwd.txt>
driver = passwd
}
Then enable and start the dovecot service:
$ doas rcctl enable dovecot
$ doas rcctl start dovecot
You now have an IMAP server running. So on any device, you can add a new IMAP Mail account, with these settings:
- Account type: IMAP
- Email address: yourusername@yourdomain.name
- Username: yourusername
- Password: the password you made for your username on your server
- Incoming mail server: yourdomain.name
- Outgoing mail server: yourdomain.name
- Connection security: SSL
- Authentication type: Basic authentication
Thanks for reading! Hope it was useful for you. Please email me at et@erict.org if you have any comments.