# Background jobs

> AppointMe runs recurring work as Hangfire jobs stored in SQL Server. Each job runs as the system and invokes Wolverine commands, usually one per company, so the .NET handlers stay tenant-scoped.

Source page: https://appointme.dev/docs/concepts/background-jobs/

* **Scheduler:** Hangfire 1.8.25, cron recurring jobs
* **Storage:** SQL Server, `hangfire` schema
* **Runs as:** `SystemIdentity` and `SystemPrincipal`
* **Dashboard:** `/admin/jobs`, super admins only

## How a recurring job runs

**Modules declare jobs through a small interface in Shared; the API host maps it to Hangfire and runs every job as the system.**

*Figure 1. Hangfire owns the schedule. The work itself is an ordinary Wolverine handler.*

1. **IRecurringJobRegistrar** — one per job, called at startup
2. **Hangfire server** — fires the cron, stores state in `hangfire`
3. **Job.Run** — as `SystemPrincipal`, loops over companies
4. **Wolverine handler** — `InvokeForCompany` runs it for one tenant

| Piece                           | Where                                                      | What it does                                                                                 |
| ------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `IRecurringJobRegistrar`        | `AppointMe.Shared/Jobs`                                    | `Register(IRecurringJobScheduler)`. One per job, next to the job in its slice.               |
| `IRecurringJobScheduler`        | `AppointMe.Shared/Jobs`                                    | `AddOrUpdate<TJob>(jobId, methodCall, cronExpression)`. Modules never reference Hangfire.    |
| `AddRecurringJobs(assembly)`    | `AppointMe.Shared/Jobs/JobsServiceCollectionExtensions.cs` | Registers every registrar in the assembly as a singleton (Scrutor). CRM and Booking call it. |
| `HangfireRecurringJobScheduler` | `AppointMe.Api/Hangfire`                                   | Forwards to Hangfire’s `IRecurringJobManager.AddOrUpdate`.                                   |
| `RecurringJobsHostedService`    | `AppointMe.Api/Hangfire`                                   | On startup, passes the scheduler to every registrar.                                         |
| `SystemContextJobFilter`        | `AppointMe.Api/Hangfire`                                   | A server filter that sets the system identity and principal for the job’s duration.          |
| `AddAppointMeHangfire`          | `AppointMe.Api/Hangfire/HangfireExtensions.cs`             | Storage, server, filter and scheduler. `Program.cs` skips it during code generation.         |

## In the code: registrar, job, handler

**A job is a plain class with a `Run` method. Its registrar gives it an id and a cron expression.**

src/Booking/AppointMe.Booking/Attendees/ReconcileAttendees/AttendeeReconciliationJobRegistrar.cs

```cs
internal sealed class AttendeeReconciliationJobRegistrar : IRecurringJobRegistrar
{
    private const string JobId = "booking:attendee-reconciliation";
    private const string CronExpression = "0 3 * * *"; // daily at 03:00


    public void Register(IRecurringJobScheduler scheduler)
    {
        scheduler.AddOrUpdate<AttendeeReconciliationJob>(
            JobId,
            job => job.Run(CancellationToken.None),
            CronExpression);
    }
}
```

Hangfire stores the expression, and when the job runs it passes its own token in place of `CancellationToken.None` ([Hangfire docs](https://docs.hangfire.io/en/latest/background-methods/using-cancellation-tokens.html)). It builds the job from DI, and `BookingModule` registers each job with `AddScoped`. The job does no business work itself: it finds the companies and invokes a command for each one.

src/Booking/AppointMe.Booking/Attendees/ReconcileAttendees/AttendeeReconciliationJob.cs

```cs
public sealed class AttendeeReconciliationJob(BookingDbContext dbContext, IMessageBus bus)
{
    public async Task Run(CancellationToken cancellationToken)
    {
        var companies = await dbContext.BookingCompanies.ToListAsync(cancellationToken);
        foreach (var company in companies)
        {
            await bus.InvokeForCompany(company.Id, new ReconcileAttendeesCommand(), cancellationToken);
        }
    }
}
```

### Who the job runs as

A job has no user and no company. `SystemContextJobFilter` pushes the system context before the job body runs, and disposes both scopes in `OnPerformed`:

src/AppointMe.Api/Hangfire/SystemContextJobFilter.cs

```cs
public void OnPerforming(PerformingContext context)
{
    var identityScope = currentIdentity.Change(new SystemIdentity());
    var principalScope = currentPrincipal.Change(new SystemPrincipal());


    context.Items[StateKey] = new State(identityScope, principalScope);
}
```

`SystemPrincipal` has every permission, so `principal.Require(...)` passes in anything the job invokes.

The company comes from `bus.InvokeForCompany(companyId, command)`. It calls Wolverine’s `InvokeForTenantAsync`, and `TenantContextPolicy` sets `ICurrentCompany` before the handler runs, so a `CompanyId` parameter and the EF Core query filters work as in a request. See [Multi-tenancy](/docs/concepts/multi-tenancy/#background-work-run-it-for-a-company) and [Identity & principal](/docs/concepts/identity-and-principal/).

## Jobs that ship

**Three reconciliation jobs repair Booking’s copies of other modules’ data. Two demo jobs keep the demo tenant’s dashboard populated.**

| Job id                                    | Class                              | Cron                     | What it does                                                                                                                   |
| ----------------------------------------- | ---------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `booking:company-reconciliation`          | `BookingCompanyReconciliationJob`  | `0 * * * *`, hourly      | Invokes `ReconcileBookingCompaniesCommand`, which upserts a `BookingCompany` for every company in `ICompanyRehydrationSource`. |
| `booking:service-provider-reconciliation` | `ServiceProviderReconciliationJob` | `0 * * * *`, hourly      | Per company, compares service providers with `IEmployeeRehydrationSource` and creates, updates or deletes to match.            |
| `booking:attendee-reconciliation`         | `AttendeeReconciliationJob`        | `0 3 * * *`, 03:00 daily | Per company, pages through CRM customers 100 at a time through `ICustomerRehydrationSource` and upserts attendees.             |
| `booking:top-up-demo-appointments`        | `TopUpDemoAppointmentsJob`         | `0 3 * * *`, 03:00 daily | Adds 15 appointments per company. Registered only when `Demo:Enabled` is true.                                                 |
| `crm:top-up-demo-customers`               | `TopUpDemoCustomersJob`            | `0 3 * * *`, 03:00 daily | Adds 2 customers per company. Registered only when `Demo:Enabled` is true.                                                     |

Cron times are UTC, Hangfire’s default, and `IRecurringJobScheduler` has no time-zone parameter. The reconciliation handlers catch and log a failure per row, then carry on with the next one. The rehydration sources are interfaces in the publisher’s `*.Contracts` project, so Booking reads Organizations and CRM data without referencing those modules.

## Dashboard and storage

**The Hangfire dashboard is at `/admin/jobs`, behind an ASP.NET Core policy that admits only super admins.**

src/AppointMe.Api/Hangfire/HangfireExtensions.cs

```cs
public IEndpointConventionBuilder MapAppointMeHangfireDashboard()
{
    return endpoints
        .MapHangfireDashboard("/admin/jobs", new DashboardOptions { Authorization = [] })
        .RequireAuthorization(HangfireDashboardPolicy.Name);
}
```

| Setting                | Value                                                                                                                                          |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| Who gets in            | `HangfireDashboardPolicy` needs an authenticated user and a `SuperAdminRequirement`: the user’s email must be in `Authentication:SuperAdmins`. |
| Hangfire’s own filters | `Authorization = []`, so the ASP.NET Core policy is the only check.                                                                            |
| Local super admin      | `demo@appointme.dev`, set in `appsettings.Development.json`                                                                                    |
| Link in the SPA        | “Background Jobs” under Administration, shown when `GET /api/v1/me` returns `isSuperAdmin: true`                                               |
| Storage                | `UseSqlServerStorage` on the `AppointMeSql` database, schema `hangfire`, created at startup (`PrepareSchemaIfNecessary = true`)                |
| Serialization          | `UseSimpleAssemblyNameTypeSerializer()` and `UseRecommendedSerializerSettings()`                                                               |

Super admin is a platform role, not a company role: `SuperAdminRegistry` is an email list, the same for every tenant. `infra/README.md` lists replacing it with a role or permission once roles reach the token.

License

Hangfire is dual-licensed, LGPL-3.0 or commercial. `THIRD-PARTY-NOTICES.md` explains that using it as an unmodified NuGet package, as AppointMe does, is allowed in commercial apps, and that obligations start only if you modify Hangfire’s own source.

## Hangfire job or Wolverine message?

**Use Hangfire for work on a clock, and a Wolverine message for work that follows from a change.**

| You need                                               | Use                                                                                                     | Why                                                                                                                                                                             |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| A recurring schedule (hourly, nightly)                 | A Hangfire recurring job                                                                                | Cron, a dashboard with history and **Trigger now**, and automatic retries. Every recurring job in AppointMe is built this way.                                                  |
| Work for every company                                 | A Hangfire job that calls `InvokeForCompany` per company                                                | It’s how the shipped jobs keep each handler tenant-scoped.                                                                                                                      |
| A one-off delay after a command (“remind in 24 hours”) | A Wolverine scheduled message: `bus.ScheduleAsync(message, delay)`, or return `message.DelayedFor(...)` | It’s stored in the durable inbox and leaves through the outbox with your transaction. A handler that takes `CompanyId` stamps the company on it. Nothing in AppointMe uses one. |
| A reaction to a change in another module               | A domain event                                                                                          | See [Domain events & messaging](/docs/concepts/messaging/).                                                                                                                     |

## Extend it

| Step                 | Where                                        | What                                                                                                                                              |
| -------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| 1. Write the command | `<Feature>/<Name>Command.cs` and its handler | The handler takes `CompanyId companyId` and does the work for one company.                                                                        |
| 2. Write the job     | `<Feature>/<Name>Job.cs`                     | `public async Task Run(CancellationToken cancellationToken)`: list the companies, call `bus.InvokeForCompany(...)` for each.                      |
| 3. Register it       | The module’s `Add<Module>Module()`           | `.AddScoped<MyJob>()`                                                                                                                             |
| 4. Schedule it       | `<Feature>/<Name>JobRegistrar.cs`            | Implement `IRecurringJobRegistrar` and call `scheduler.AddOrUpdate<MyJob>("module:my-job", job => job.Run(CancellationToken.None), "0 * * * *")`. |
| 5. Discover it       | The module’s `Add<Module>Module()`           | `.AddRecurringJobs(ModuleAssembly.Instance)`, if the module doesn’t call it yet                                                                   |
| 6. Check             | `/admin/jobs`                                | Find it under Recurring jobs and click **Trigger now**.                                                                                           |

## Gotchas

| Symptom                                                                     | Cause                                                                                                                                                        | Fix                                                                                                     |
| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
| A job you deleted, or switched off with `Demo:Enabled`, still runs          | Recurring jobs persist in the `hangfire` schema. A registrar that skips `AddOrUpdate` doesn’t remove one, and `IRecurringJobScheduler` has no remove method. | Delete it in the dashboard, or call `IRecurringJobManager.RemoveIfExists(jobId)`.                       |
| A job fails with `AccessDeniedException: Active company was not specified.` | The job used `InvokeAsync` for a handler that takes `CompanyId`                                                                                              | Use `bus.InvokeForCompany(companyId, command)`.                                                         |
| One company’s failure stops the rest                                        | `InvokeForCompany` runs inline and throws, and the shipped job loops don’t catch                                                                             | Catch per company in the job, or keep handlers idempotent so Hangfire’s retry of the whole job is safe. |
| You’re signed in but `/admin/jobs` won’t open                               | Your email isn’t in `Authentication:SuperAdmins`                                                                                                             | Add it for that environment.                                                                            |
| A job runs at the wrong local hour                                          | Cron is evaluated in UTC                                                                                                                                     | Write the expression in UTC.                                                                            |

### Why Hangfire next to Wolverine?

Hangfire brings what a recurring job needs: cron schedules, a dashboard with run history, manual triggers and retries. The jobs stay thin, because each one only dispatches Wolverine commands, so the logic lives in handlers you can call and test like any other.

Modules depend on `IRecurringJobScheduler`, not on Hangfire, so swapping the scheduler touches the API host only. The cost is a second store of background state (`hangfire` beside `wolverine`) and an LGPL dependency.

### Why reconciliation jobs, if events are durable?

Durable delivery doesn’t cover every failure. A handler bug can dead-letter events, and a module you add later starts with no copy of the past. The reconciliation jobs rebuild Booking’s projections from the source modules on a schedule, so they heal themselves without anyone replaying messages.

## Next

* [Domain events & messaging](/docs/concepts/messaging/) (Concept)
* [Multi-tenancy](/docs/concepts/multi-tenancy/) (Concept)
* [Booking](/docs/modules/booking/) (Module)