Docs · Guides

Background jobs.

Work that should not make a visitor wait — syncing stock, sending email, resizing images. On Canner a project runs one process, so the simplest setup is a job queue kept in your Postgres database that runs inside that process. This page shows it with Rails and GoodJob.

Run the queue in the web process

GoodJob stores jobs in your Canner database and, in its async mode, runs them in threads of the Rails server itself. That is the setup Rails and GoodJob recommend for a single server, and it needs nothing else from Canner: the jobs use the same code, the same environment variables and the same database as the web app, and they restart with it.

# config/environments/production.rb  (or set GOOD_JOB_EXECUTION_MODE=async)
config.active_job.queue_adapter = :good_job
config.good_job.execution_mode = :async   # jobs run in threads of the web process
config.good_job.max_threads = 3

Add the good_job gem, run its install generator to get the migration, and apply it with your release command:

bundle exec rails db:migrate

Jobs start as soon as they are enqueued: GoodJob listens for new jobs through Postgres, so there is no polling delay. In our tests on Canner, jobs started within tens of milliseconds, and 200 jobs enqueued at once each ran exactly once.

Database connections

Your app’s DATABASE_URLgives each connection its own Postgres session. Things that need a session — advisory locks, LISTEN/NOTIFY, prepared statements, SET — work, which is what GoodJob relies on. Up to 20 connectionsper database can be open at once, so size the pool to what you use: web threads, plus job threads, plus 3 for GoodJob’s listener and housekeeping.

# config/database.yml
production:
  url: <%= ENV["DATABASE_URL"] %>
  # web threads (RAILS_MAX_THREADS) + job threads + 3 for GoodJob itself
  pool: <%= ENV.fetch("DB_POOL", 12) %>

Memory

Jobs share the project’s memory with the web server, and a heavy job also shares its processor time, which can slow requests while it runs. Watch the memory line on the project overview after your first real jobs and raise the project’s memory in its settings if it comes close to the limit (see plans and limits).

Sleep, crashes and deploys

  • Sleep. On Starter, a project with no visitors for 3 hours goes to sleep, and its jobs stop with it. The next request wakes it, and so does a scheduled job that calls your app. To run jobs around the clock, keep the project always-on, which Live includes for two projects.
  • Crashes. If the process stops, it is started again after a few seconds.
  • Deploys. The old version gets 30 seconds to finish before it is stopped. GoodJob keeps jobs in the database, so a job cut off by a deploy is retried by the new version; keep jobs shorter than that, or make them safe to run twice.

A separate worker process

A project runs one process, so a second, always-on worker next to the web server is not supported yet. If you need isolated memory for your jobs, tell us what you are running.