Zero-downtime deployments

Ponder supports zero-downtime deployments using health checks. This guide describes how to enable zero-downtime deployments for both HTTP (GraphQL) requests and direct SQL queries.

Healthchecks

The /health route returns 200 once the app becomes healthy, and 503 before. This route can be used by load balancers and platform services to determine if an instance is ready to receive traffic.

Here is the logic Ponder users to decide if it should respond as healthy:

• If historical indexing is not complete, and the process uptime is less than maxHealthcheckDuration seconds, respond as not healthy.
• If historical indexing is not complete, but the process uptime is greater than maxHealthcheckDuration seconds, respond as healthy.
• If historical indexing is complete, respond as healthy.

The maxHealthcheckDuration option is configurable, and defaults to 240 seconds (4 minutes).

HTTP / GraphQL with zero downtime

To ensure zero downtime for GraphQL queries across redeployments, only begin routing GraphQL HTTP requests to an instance once it is healthy.

Ponder should work out-of-the box with most deployment tools that support health checks, including platform-as-a-service offerings like Railway and Render as well as container orchestration tools like Kubernetes.

Direct SQL with zero downtime

Ponder supports zero-downtime deployments for direct SQL queries by creating or updating views in a publish schema that serve data from the latest healthy deployment. The deployment-specific schemas (which contain the actual indexed data tables) have an unstable name that is unique to each deployment. Read more about database schema isolation.

When using this pattern, you should run SQL queries against the publish schema (stable name), not deployment-specific schemas (unstable names).

To enable publishing, set the publishSchema option in ponder.config.ts.

import { createConfig } from "@ponder/core"
 
export default createConfig({
  database: {
    kind: "postgres",
    publishSchema: "indexer", // Default: undefined
  },
});

In a zero-downtime deployment scenario, it usually makes sense to set schema to a unique, deployment-specific value and set publishSchema to "public". With this configuration, the views in the publish schema always serve data from the latest deployment to become healthy.

Example scenario (Railway)

To improve the experience on Railway, if Ponder detects the RAILWAY_DEPLOYMENT_ID environment variable, it sets publishSchema to "public".

Here's how zero-downtime for direct SQL works on Railway.

1. You create a new Postgres database service for the app.
2. You create a new service named indexer linked to the GitHub repository of your Ponder app. Once created, Railway initiates the first deployment for this service.
3. When the first deployment starts up, it creates and begins indexing in an isolated schema named ${RAILWAY_SERVICE_NAME}-${RAILWAY_DEPLOYMENT_ID.slice(0,8)}, e.g. indexer-87cbd10a.
4. When the deployment becomes healthy, it creates creates views in the "public" schema that serve data from the tables in indexer-87cbd10a.
5. You push a commit to your repository, which triggers a new deployment. The new deployment starts indexing in an isolated schema, this time named indexer-9b3c4d21.
6. When the 2nd deployment becomes healthy, it drops the views in the "public" schema and creates new ones that serve data from indexer-9b3c4d21.

Using this scheme, the views in the "public" schema always serve data from the latest deployment. This mirrors the behavior of the public GraphQL endpoint for the service.

Updating ponder.schema.ts

If you deploy a version of your app that makes a breaking change to ponder.schema.ts, you should first ensure that any downstream services are compatible with both the old and new column definitions. As soon as the new deployment becomes healthy, it will replace the views in the publish schema using the new column definitions. If you're not careful, this process will break downstream services and may cause downtime.