Live Background Job Progress in Rails 8.1 with Rails.event and Continuations

· 11 min read

Showing users a real-time progress card while a background job runs has always been more work than it should be. You need a place to store progress, code in the job that calls broadcast_replace_to after every chunk, a partial that renders the card, an Action Cable channel, and the discipline to wire all four together for every job that needs progress. Most teams skip it.

Rails 8.1 added the three primitives that turn this from “wire four things together for every job” into something much smaller: a structured event reporter (Rails.event), framework-emitted structured events for every Active Job lifecycle stage, and ActiveJob::Continuable for cursor-tracked multi-step jobs. Connect them and you get user-facing live progress with no broadcast_* calls in the job body and no per-job wiring.

This post walks through the pattern, with verifiable code in a companion repo.

What Rails 8.1 ships for free

Three pieces, all new in Rails 8.1.0:

Rails.event is a structured event reporter. You emit events with Rails.event.notify("user.signup", user_id: 123) and register subscribers with Rails.event.subscribe(subscriber). Each subscriber receives a hash with name, payload, tags, context, timestamp, and source_location. Rails.event sits on top of ActiveSupport::Notifications: the framework’s structured event subscribers consume legacy notifications and re-emit them with the richer schema, so for new structured observability code, subscribing to Rails.event alone is sufficient. Legacy tools that already subscribe to ActiveSupport::Notifications (APMs, Skylight, and so on) keep working unchanged.

Structured Active Job events are emitted by Rails itself for every lifecycle stage. For every job: active_job.enqueued, active_job.bulk_enqueued, active_job.started, active_job.completed, active_job.retry_scheduled, active_job.retry_stopped, active_job.discarded. For Continuation jobs you also get active_job.step_started, active_job.step, active_job.step_skipped, active_job.interrupt, and active_job.resume. They all flow through Rails.event.notify, so the same subscribers see them.

ActiveJob::Continuable lets you split a job into named step blocks with optional cursors. The framework persists progress so the job can be interrupted and resumed across worker restarts, deploys, and retries.

The framework lifecycle and step events handle the coarse states a UI needs (queued, running, step boundaries, done, failed) without any code in the job. For mid-step progress like “imported 47 of 200,” your job emits a small custom event inside the loop. The example below does both.

The job

Notice what is not in the body below: no broadcast_replace_to, no update! on a tracking record, no progress percentage math. The only thing that resembles progress instrumentation is step.advance!, which Continuations need for resumability anyway.

class ImportContactsJob < ApplicationJob
  include ActiveJob::Continuable

  before_perform { Rails.event.set_context(user_id: arguments.first, job_id: job_id) }

  def perform(user_id, rows:)
    step :prepare { Rails.event.notify("import.prepared", total: rows.size) }
    step :process_records do |s|
      start = s.cursor || 0
      rows[start..].each_with_index do |row, i|
        Contact.create!(row.merge(user_id: user_id))
        Rails.event.notify("import.record_processed", index: start + i)
        s.advance! from: start + i
      end
    end
    step :finalize { Rails.event.notify("import.finalized") }
  end
end

The before_perform callback is the load-bearing piece. It puts user_id and job_id into ambient context for the rest of the execution, so every event the job emits afterwards (including the framework lifecycle events) carries them. The subscriber uses user_id to pick which stream to broadcast to and job_id to pick which DOM element to update.

You do not need to clear the context. Rails wraps each job in app.reloader.wrap, and the executor’s to_complete hook calls Rails.event.clear_context automatically.

The subscriber

One subscriber, registered once. It receives both the framework lifecycle events and your custom import.* events through the same channel.

# config/initializers/job_progress_subscriber.rb
Rails.event.subscribe(JobProgressBroadcaster.new) do |event|
  event[:context][:user_id].present? &&
    (event[:name].start_with?("active_job.") || event[:name].start_with?("import."))
end

The filter block keeps the subscriber from seeing the whole application’s event stream. Only events under the active_job.* or import.* namespaces that also carry a user_id in context reach emit.

The broadcaster itself is just a case over event names that turns each one into a Turbo Stream broadcast. The shape:

class JobProgressBroadcaster
  def emit(event)
    user_id = event[:context][:user_id]
    job_id  = event[:payload][:job_id] || event[:context][:job_id]
    update  = translate(event[:name], event[:payload])
    return unless update && user_id && job_id

    Turbo::StreamsChannel.broadcast_replace_to(
      "user_#{user_id}_jobs",
      target: "job_#{job_id}",
      partial: "jobs/progress_card",
      locals: { update: update }
    )
  end
end

translate returns a hash for the events you care about (active_job.started, import.prepared, import.record_processed, active_job.completed, active_job.discarded) and nil for everything else. The full case is in the companion repo.

The view

The page that should display progress mounts the stream and renders the initial card:

<%= turbo_stream_from "user_#{current_user.id}_jobs" %>
<div id="job_<%= @job_id %>"><%= render "jobs/progress_card", update: { status: "queued" } %></div>

_progress_card.html.erb renders whatever shape your design calls for from the update hash. This assumes your ApplicationCable::Connection is identified by current_user: turbo_stream_from signs the stream name with Rails’ message verifier so it cannot be tampered with, but the per-user security model relies on Action Cable knowing who the connecting user is.

That is the entire user-facing pipeline. When ImportContactsJob runs, the page updates in real time. No polling, no hand-written JavaScript, no per-job Cable channel.

What you get for free

Three claims worth verifying. The companion repo has a passing test for each.

Lifecycle events. active_job.enqueued, active_job.started, and active_job.completed all flow through Rails.event with matching job_id and a duration field on completed. Your subscriber sees the queued/running/done transitions without any code in the job body.

Step events with cursor. Each step declaration produces an active_job.step_started event (with cursor and resumed fields) and an active_job.step event (with the final cursor and duration). Resumed jobs have resumed: true on the step that was in progress, so the UI can show “Resuming where we left off.”

Context from before_perform. Every event emitted after the before_perform callback runs, both your custom events and the framework events, carries the context you set. That is how the subscriber knows which user’s stream to broadcast to.

Caveats

A few things the source code makes clear once you go looking.

Cursor semantics. step.advance!(from: x) calls x.succ, so the cursor reflects “next position to start from on resume,” not “last index processed.” If you write s.advance! from: i + 1 after processing index i, the cursor ends up at i + 2, not i + 1. The canonical pattern from the Rails docs is s.advance! from: record.id which sets the cursor to record.id.succ, the next ID find_each(start: cursor) will pick up. For percentage display, prefer counting in your own progress event (Rails.event.notify("import.record_processed", index: i)) rather than reading the cursor.

The enqueued event fires on the enqueueing thread. active_job.enqueued fires the moment perform_later returns, on whatever thread called it (your controller, usually). active_job.started and everything after fires on the worker thread. With the :async adapter or a real queue adapter, this is what you want. With the :inline adapter, the whole job runs synchronously inside perform_later, so started and completed fire before enqueued. Worth knowing if you ever debug this in test or a console.

Because enqueued fires before before_perform runs, it does not carry the context the job will eventually set. Its event[:context] is whatever the enqueueing thread had, which is typically empty. The filter above will exclude it. If you want a “queued” broadcast for the user, render the queued state from the controller after perform_later returns (the view example does this), or set Rails.event.set_context(user_id: current_user.id, job_id: job.job_id) in the controller right before enqueuing.

arguments.first is positional. The before_perform block reads arguments.first to grab user_id. If you ever change the perform signature so user_id is not the first positional argument, update the context wiring at the same time or the subscriber will broadcast for the wrong user. A keyword-only perform(user_id:, rows:) with arguments.first[:user_id] is more durable in larger codebases.

Adapter-agnostic, but timing is not. SolidQueue, GoodJob, Sidekiq, and the async adapter all emit the same events because the events come from Active Job, not from the adapter. A subscriber that works on one works on all. The latency between enqueued and started, however, depends on worker availability, not Rails.

Multi-tenant filtering. The user_id in context is what scopes broadcasts to a single user. If your subscriber is doing anything sensitive (sending notifications, billing the user), keep your filtering inside the subscriber and not in the view. A bug in the filter is much easier to spot in one initializer than across every page.

Long-running jobs and presence. Action Cable does not replay broadcasts that fired while the user was disconnected. If someone closes the tab mid-job and reopens it later, they will see nothing until the next event fires, and if the job has already completed they will see nothing at all. Persist a TrackedJob row updated by the same subscriber so the page renders the current state on first load and the stream just delivers updates on top.

FAQ

What is Rails.event?

Rails 8.1 added a structured event reporter at Rails.event. You call Rails.event.notify(name, payload) to emit a typed event, register subscribers with Rails.event.subscribe(subscriber), and they receive a hash with name, payload, tags, context, timestamp, and source_location. It sits on top of ActiveSupport::Notifications: the framework’s structured event subscribers consume legacy notifications and re-emit them with the richer schema, so for new structured observability code subscribing to Rails.event alone is sufficient. Legacy tools that already subscribe to ActiveSupport::Notifications keep working unchanged.

Does this require Rails 8.1?

Yes. Rails.event ships in Rails 8.1.0 (released 2025-10-22). The structured Active Job events (active_job.enqueued, started, completed, step_started, step) also ship in 8.1. ActiveJob::Continuable and the step DSL are also Rails 8.1 features.

Does this work with Sidekiq, SolidQueue, GoodJob?

Yes. The events flow through Rails.event regardless of which adapter runs the job. You write one subscriber and it works across every adapter.

Do users have to refresh the page to see progress?

No. Pair the subscriber with Turbo::StreamsChannel.broadcast_replace_to to push updates over Action Cable. The user’s browser updates the progress card in place as events arrive.

What happens if the worker process dies mid-job?

Continuations checkpoint progress to the job’s serialized state. When a worker picks up the retried job, completed steps are skipped and the in-progress step resumes from its last cursor. The active_job.step_started event for the resumed step has resumed: true so the UI can show “Resuming where we left off” if you want.

Wrapping up

The pieces individually are not new ideas. Action Cable broadcasts, progress columns on a tracking model, find_each with cursors. What is new in Rails 8.1 is that the three primitives you need (Rails.event, structured Active Job events, Continuations) all line up, so the integration is a few small files instead of a few hundred lines of broadcast plumbing scattered across every job.

If you want to verify any of the claims here against shipped Rails 8.1.3, the rails-event-jobs-spike repo has a bin/run two-thread streaming demo and five passing integration tests. The original PR for Rails.event is rails/rails#55334, and the announcement is on the Rails blog.