gem-hunterDBLab enables 🖖 database branching and ⚡️ thin cloning for any Postgres database and empowers DB testing in CI/CD. This optimizes database-related costs while improving time-to-market and software quality. Follow to stay updated.
*For managed PostgreSQL cloud services like AWS RDS or Heroku, direct physical connection and PGDATA access aren't possible. In these cases, DBLab should run on a separate VM within the same region. It will routinely auto-refresh its data, effectively acting as a database-as-a-service solution. This setup then offers thin database branching ideal for development and testing.
For example, cloning a 1 TiB PostgreSQL database takes just about 10 seconds. On a single machine, you can have dozens of independent clones running simultaneously, supporting extensive development and testing activities without any added hardware costs.
Try it yourself right now:
demo-token to access)Thin cloning is fast because it is based on Copy-on-Write (CoW). DBLab employs two technologies for enabling thin cloning: ZFS (default) and LVM.
Using ZFS, DBLab routinely takes new snapshots of the data directory, managing a collection of them and removing old or unused ones. When requesting a fresh clone, users have the option to select their preferred snapshot.
Read more:
databaseRename option)retention config; safe-only, never force-deletes dependents)/healthz API endpoint (no auth), extended /status endpoint (API docs)/metrics) for monitoringA single clone can be moved to a newer PostgreSQL major version without touching the rest of the instance, which makes it practical to test an upgrade against production-like data:
dblab clone upgrade my-clone
The target version is not a parameter: it follows from the upgrade image the instance is
configured with, which is built on that major's binaries. The engine reports it in
GET /status as cloneUpgrade.targetVersion, the CLI prints it, and the UI shows it on the
confirmation dialog.
The same action is available in the UI on the clone page and over the API as
POST /clone/{id}/upgrade.
The clone's Postgres is shut down cleanly, pg_upgrade --link converts its data directory in
place, and the clone restarts on an image of the target major. Because --link hard-links the
data files instead of copying them, the upgrade is fast and consumes almost no extra space. The
request returns as soon as the clone enters the UPGRADING state; watch the clone status for the
result.
Enabling it. Set provision.pgUpgradeImage to an upgrade image of the major you want clones
upgraded to. That image is the single source of the target: the engine reads the major out of a
release tag (:17, :17-0.8.0, :17-0.8.0-glibc236), and re-reads it from the image's own
PG_MAJOR before every upgrade, so a tag repointed at another build is caught before any data
moves. A tag outside that grammar — a digest pin, latest, a CI build — states no major, so the
engine fetches the image in the background at startup and reads it from there; until that lands
the upgrade reports itself unavailable. When pgUpgradeImage is unset the feature is simply
unavailable and the endpoint says so; nothing else about the instance changes. The UI keeps the
"Upgrade clone" button visible but disabled in that state, and its tooltip carries the reason
reported on /status, so an unset image is one hover away from being diagnosed.
provision: pgUpgradeImage: "postgresai/pg-upgrade:17" pgUpgradeTimeout: 3h pgUpgradePullTimeout: 1h
pgUpgradeTimeout bounds a single upgrade run; it defaults to three hours. When the budget runs
out, the upgrade container is removed and the clone is put back on the version its data directory
still holds, exactly as it would be after any other failure.
pgUpgradePullTimeout bounds each of the two image pulls that precede it, and defaults to one
hour. The upgrade image carries the target major plus the server packages of the four preceding
ones, so the first upgrade on an instance downloads several gigabytes; pre-pulling both images
makes the step a no-op. Both pulls happen while the clone is still serving traffic, so exceeding
the budget leaves the clone untouched and running. The two budgets are separate on purpose — a
slow pull must not eat into the time pg_upgrade gets once the clone is already stopped.
An upgrade request may name an image explicitly instead of letting the engine substitute the major
in the current tag. provision.upgradeImageAllowList says which repositories such a request may
name. When it is empty, as it is by default, only the repository of the clone image is accepted:
the one databaseContainer.dockerImage points at, plus the repository of the clone's own image
when the clone has already been upgraded to another one. That default accepts exactly the image
the engine would derive itself. List repositories to allow other ones, or "*" to allow any
repository the instance can reach.
provision: upgradeImageAllowList: - "postgresai/extended-postgres"
Releases up to v4.2.0 accepted any repository when the list was empty. An instance that relied on
that must now set the list to ["*"].
How it can end. Almost every outcome leaves the clone running:
| Status | Meaning |
|---|---|
OK |
The clone runs the target version; dbVersion reports it. |
WARNING |
The upgrade was not applied. Either nothing had been converted and the clone still runs its original version, or conversion had begun and the clone was rebuilt from its origin snapshot — in which case data written since the clone was created is lost. The message says which, and points at the pg_upgrade log under <clone dir>/upgrade/logs/. When the postmaster that pg_upgrade started failed, its own fatal error, with the DETAIL and HINT lines that follow it, is appended to the cause, and the run's pg_upgrade_output.d/<timestamp>/ directory (server log, internal log, schema dumps) is preserved under the same logs/ directory; the five most recent failed runs are kept. |
WARNING, clone not running |
The upgrade settled on disk but the clone container did not come back up. The data directory is intact and the upgrade is still recorded as pending, so the next engine start finishes or undoes it and brings the clone back. |
FATAL |
Recovery itself failed and there is nothing left to finish. Reset or delete the clone. |
Rolling back. Resetting the clone returns it to the version the instance is configured with, because a reset re-provisions from the origin snapshot using the engine-wide image.
Things to know.
pgUpgradeImage at a different major.…:16-0.8.0-glibc236 → …:17-0.8.0-glibc236). Changing the glibc build across an upgrade
would change collation behaviour, and pg_upgrade does not reindex. Pass --docker-image to
override the choice; when its tag names a major, that major has to be the one being upgraded to.pg_upgrade verifies that every extension in the source database has a matching library for
the new major. An extension outside the image's set fails this check, and the clone rolls back
untouched.dockerImage must still match the source major.The simplest way to show your support is by giving us a star on GitHub or GitLab! ⭐
If DBLab has been a vital tool for you, tell the world about your journey. Use the logo from the ./assets folder for a visual touch. Whether it's in documents, presentations, applications, or on your website, let everyone know you trust and use DBLab.
HTML snippet for lighter backgrounds:
<a href="http://databaselab.io"> <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-light-background.svg" /> </a>
For darker backgrounds:
<a href="http://databaselab.io"> <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-dark-background.svg" /> </a>
For proposals, bug reports, and participation in development, see our Contributing Guide.
You can find more in the "How-to guides" section of the documentation.
The DBLab source code is licensed under the OSI-approved open source license Apache 2.0.
Reach out to the PostgresAI team if you want a trial or commercial license that does not contain the GPL clauses: Contact page.
Many thanks to our amazing contributors!
Making DBLab more accessible to engineers around the globe is a great help for the project. Check details in the translation section of contributing guide.
This README is available in the following translations: