#go/pgcluster

#go/pgcluster

Package pgcluster boots an ephemeral PostgreSQL cluster for a test binary.

This is a cluster launcher, one layer below the per-test database managers consumers already have: it boots a real postmaster on a private tmpfs directory with fsync off, hands back a base libpq URL, and shuts the cluster down again. Creating and dropping a database per test is then an ordinary CREATE DATABASE against that URL.

The model is one shared cluster per test binary, many ephemeral databases inside it. Booting a cluster costs roughly a second; creating a database inside a running one costs milliseconds. A per-test cluster would be neither.

#Usage

The shared cluster belongs in TestMain, where it can be started once and stopped once. [Start] is TB-free for exactly this reason, and its errors wrap sentinels so a machine without PostgreSQL skips instead of failing:

var cluster *pgcluster.Cluster

func TestMain(m *testing.M) { started, err := pgcluster.Start("MYAPP_DATABASE_URL") switch { case errors.Is(err, pgcluster.ErrPostgresUnavailable): // Leave cluster nil; each test skips with err.Error(). case err != nil: fmt.Fprintln(os.Stderr, err) os.Exit(1) default: cluster = started } code := m.Run() if cluster != nil { cluster.Stop() } os.Exit(code) }

A test then takes a database of its own, which is created, exported under the DSN variable, and dropped again when the test ends:

func TestSomething(t *testing.T) { url := cluster.Database(t) // MYAPP_DATABASE_URL now names a fresh empty database. }

[Ephemeral] is the TB-bound alternative for a suite that wants a whole cluster scoped to one test (or to one subtest) rather than to the binary. It fails the test rather than returning an error, and registers its own shutdown through TB.Cleanup.

#The DSN variable is mandatory

There is no default for dsnEnv: the variable an application reads its connection string from is the application's decision, and guessing it would either do nothing or silently point production configuration at a test cluster.

#Zero dependencies

The package depends only on the standard library and the PostgreSQL binaries it launches -- initdb, pg_ctl and psql. It drives them as subprocesses and never links a driver, so adopting the test floor cannot drag a database driver into a consumer's module graph.

#A killed test binary leaks its postmaster

Read this before adopting the package. [Cluster.Stop] runs from TB.Cleanup or from TestMain, and both are ordinary Go code: neither runs when the test binary dies without unwinding -- SIGKILL, an editor or CI job cancelling the run, an OOM kill. When that happens the postmaster stays alive with its data directory deleted underneath it, and its socket directory survives as a stray.

This is accepted, not solved. The Python implementation of the same launcher registers an atexit hook, which covers the interpreter's ordinary exit paths. Go has no equivalent: there is no runtime hook that fires on process exit, and the substitutes are worse than the leak. A signal handler cannot catch SIGKILL (the case that actually happens), and installing one would fight the consuming suite for the same signals; a watchdog goroutine dies with the process it is watching.

The recorded alternative, not built: a PID-file reaper. Each cluster would record its postmaster PID and directories in a well-known registry file, and every later start would sweep the registry, killing any recorded postmaster whose data directory no longer exists and removing its leftovers. That turns an unbounded leak into one cleaned up at the next test run. It is a real design with real hazards (a stale PID can be reused by an unrelated process, and the registry becomes shared mutable state between concurrent test binaries), so it is written down here rather than shipped by default.

Until then: the strays are visible and cheap to clear. Every directory this package creates is named stpg-*, so pkill -f stpg- ends any orphaned postmaster and rm -rf /dev/shm/stpg-* removes what it left behind. They live on a tmpfs, so a reboot clears them too.

#No in-process network guard can see a C driver

The cluster listens on a unix socket and nothing else, so it cannot collide with a real local server and cannot be reached from off the machine. That is the whole of the isolation this package provides, and it is worth being exact about what it is not.

Go suites have no in-process network guard at all -- see the hygiene package's documentation for why one is not shipped -- so nothing intercepts a Go driver's dial in either direction. It is worth stating that even where such a guard exists it would not help: the Python plugin's socket guard is built on sys.addaudithook and sees only connects made through Python's socket module, so a libpq-backed driver (psycopg) opens its connection in C, entirely outside the guard's view. Adding the socket directory to an allowlist changes nothing for such a driver, because there is no event to allow.

The protection that does work is the same one in every language: point the application's DSN at this cluster. A driver that connects to the URL under dsnEnv reaches a throwaway database on a private socket, whatever it is implemented in.

#SUNPathMax

const SUNPathMax = 107

SUNPathMax is the number of usable bytes in a unix socket address.

A unix socket address is a fixed-size sockaddr_un.sun_path char array. Linux gives it 108 bytes INCLUDING the NUL terminator, so 107 usable bytes; the BSDs are smaller still (104). The limit is enforced by the kernel, not by PostgreSQL, and it applies to the full socket FILE path -- the directory plus PostgreSQL's own .s.PGSQL. name. Exceeding it fails at bind() time with a message that names neither the limit nor the path, which is why this package refuses up front instead.

#ErrPostgresUnavailable

var ErrPostgresUnavailable = errors.New("PostgreSQL is not usable on this machine")

ErrPostgresUnavailable means no usable PostgreSQL installation was found, or the machine cannot host a cluster (no writable parent directory, or the process is root -- PostgreSQL refuses to run as root). The wrapped message carries the precise reason, suitable for a skip message verbatim.

#ErrSocketPathTooLong

var ErrSocketPathTooLong = errors.New("unix socket path exceeds the kernel's sun_path limit")

ErrSocketPathTooLong means a socket directory would produce a path past the kernel's sun_path limit. See [SUNPathMax].

#ErrInvalidArgument

var ErrInvalidArgument = errors.New("invalid argument")

ErrInvalidArgument means a caller-supplied value was refused before anything was executed: an empty dsnEnv, a socket directory containing whitespace, or a database name outside the closed character set.

#ErrCluster

var ErrCluster = errors.New("PostgreSQL cluster command failed")

ErrCluster means a PostgreSQL program this package ran failed. The wrapped message carries the command, its output, and the server log when one exists -- a bind() failure or a refused start says nothing useful on its own.

#Binaries

type Binaries struct

Binaries holds resolved paths to the PostgreSQL programs this package runs.

#Cluster

type Cluster struct

Cluster is a throwaway PostgreSQL cluster on a private tmpfs directory.

A Cluster is created already started, by [Start] or [Ephemeral]. It is not safe for concurrent use by multiple goroutines: every test that touches one mutates process-wide environment state through TB.Setenv, which is itself incompatible with T.Parallel.

#Option

type Option func(*Cluster)

Option customizes a cluster before it starts. Options are applied in the order they are given.

#FindBinaries

func FindBinaries(extraDirs ...string) (Binaries, error)

FindBinaries locates initdb, pg_ctl and psql.

PATH wins; otherwise the layouts in this package's search list are tried in order, with version-globbed directories sorted descending so a newer major version wins over an older one and the choice never depends on directory order. extraDirs is searched before either.

The returned error wraps [ErrPostgresUnavailable] and names exactly what was missing and where it was looked for, so a suite can skip with it verbatim.

#SocketPathFor

func SocketPathFor(dir string, port int) string

SocketPathFor is the socket file PostgreSQL will create for port in dir.

#CheckSocketDir

func CheckSocketDir(dir string, port int) (string, error)

CheckSocketDir validates a socket directory, returning the socket path it would produce.

It returns an error wrapping [ErrSocketPathTooLong] when the resulting socket path would not fit in sockaddr_un.sun_path, and one wrapping [ErrInvalidArgument] when the path contains whitespace (the directory is passed to the postmaster inside a space-separated options string, where a space would silently split it).

#GenerateName

func GenerateName() string

GenerateName returns a fresh, unique, always-acceptable database name.

#Port

func Port(port int) Option { return func(c *Cluster) { c.port = port } }

Port sets the port the postmaster listens on. It is part of the socket file name (there is no TCP listener), so it counts toward the sun_path limit.

#Superuser

func Superuser(name string) Option { return func(c *Cluster) { c.superuser = name } }

Superuser sets the name of the superuser initdb creates. It becomes the user in every URL this cluster hands out.

#MaintenanceDB

func MaintenanceDB(name string) Option { return func(c *Cluster) { c.maintenanceDB = name } }

MaintenanceDB sets the database [Cluster.BaseURL] points at and that CREATE/DROP DATABASE statements are issued from.

#DataParent

func DataParent(dir string) Option { return func(c *Cluster) { c.dataParent = dir } }

DataParent pins the directory the cluster's data directory is created under, instead of trying the built-in candidates. An explicit choice is never second-guessed: if it is unusable, starting fails.

#SocketParent

func SocketParent(dir string) Option { return func(c *Cluster) { c.socketParent = dir } }

SocketParent pins the directory the cluster's socket directory is created under, instead of trying the built-in candidates (/dev/shm, then /tmp). This is the escape route when the default parents produce a path past [SUNPathMax]. An explicit choice is never second-guessed.

#UseBinaries

func UseBinaries(b Binaries) Option

UseBinaries supplies already-resolved PostgreSQL binaries, skipping discovery. A suite that calls [FindBinaries] once (to decide whether to skip) passes the result back through this option rather than paying for the search again.

#StartTimeout

func StartTimeout(d time.Duration) Option { return func(c *Cluster) { c.startTimeout = d } }

StartTimeout sets how long pg_ctl is given to start and to stop the postmaster. It is passed through as whole seconds.

#Start

func Start(dsnEnv string, opts ...Option) (*Cluster, error)

Start initdbs into tmpfs, starts a postmaster, exports its base URL under dsnEnv, and returns the running cluster. [Cluster.Stop] shuts it down again and restores dsnEnv to whatever it held before.

Start takes no testing.TB so it can be called from TestMain, where there is none. Every error it returns wraps one of this package's sentinels; in particular [ErrPostgresUnavailable] is the "skip this suite" answer.

#Ephemeral

func Ephemeral(t testing.TB, dsnEnv string, opts ...Option) *Cluster

Ephemeral starts a cluster bound to t: it is stopped, and everything it wrote is removed, when t finishes. The base URL is exported under dsnEnv through TB.Setenv, so the testing package restores the caller's own value.

Failures fail t rather than being returned; a test that asks for a cluster outright has already decided it needs one. A suite that wants to SKIP when PostgreSQL is missing should start the cluster in TestMain with [Start] and check for [ErrPostgresUnavailable], or call [FindBinaries] first.

TB.Setenv panics under T.Parallel. That is intended: a parallel test cannot own a process-wide variable like the DSN one.

#Binaries.BinDir

func (b Binaries) BinDir() string { return filepath.Dir(b.Initdb) }

BinDir is the directory the binaries were found in.

#Cluster.Running

func (c *Cluster) Running() bool { return c.running }

Running reports whether the postmaster is up.

#Cluster.SocketDir

func (c *Cluster) SocketDir() string { return c.socketDir }

SocketDir is the directory holding the cluster's unix socket.

#Cluster.DataDir

func (c *Cluster) DataDir() string { return c.dataDir }

DataDir is the cluster's data directory.

#Cluster.SocketPath

func (c *Cluster) SocketPath() string { return SocketPathFor(c.socketDir, c.port) }

SocketPath is the full path of the cluster's unix socket file.

#Cluster.MaintenanceDB

func (c *Cluster) MaintenanceDB() string { return c.maintenanceDB }

MaintenanceDB is the database [Cluster.BaseURL] points at.

#Cluster.BaseURL

func (c *Cluster) BaseURL() string { return c.URLFor(c.maintenanceDB) }

BaseURL is the libpq URL of the maintenance database. This is what gets exported under dsnEnv, and what a per-test database manager connects to in order to CREATE DATABASE.

#Cluster.URLFor

func (c *Cluster) URLFor(dbname string) string

URLFor is the libpq URL of dbname in this cluster.

#Cluster.Stop

func (c *Cluster) Stop()

Stop stops the postmaster and deletes everything it wrote. It is idempotent, and it is safe to call on a cluster whose start failed part-way.

Nothing calls it automatically on process death: see the package documentation on the killed-binary leak.

#Cluster.CreateDatabase

func (c *Cluster) CreateDatabase(name string) (string, error)

CreateDatabase creates a database in this cluster and returns its libpq URL. An empty name gets a fresh generated one -- the ordinary per-test case.

#Cluster.DropDatabase

func (c *Cluster) DropDatabase(name string) error

DropDatabase drops a database, disconnecting anything still attached to it.

#Cluster.Database

func (c *Cluster) Database(t testing.TB) string

Database creates a fresh database for the duration of t, exports its URL under the cluster's DSN variable through TB.Setenv, and drops it when t finishes. It returns the database's URL.

This is the per-test half of the model: one cluster for the binary, one database per test. An application that reads its connection string from the environment lands in this test's own database without knowing the difference.

#Cluster.SQL

func (c *Cluster) SQL(statement string) (string, error)

SQL runs one statement against the maintenance database through psql and returns its trimmed output.

A subprocess rather than a driver on purpose: this module depends on the standard library only, and a database driver would be a dependency every consumer inherits.

#Cluster.SQLIn

func (c *Cluster) SQLIn(dbname, statement string) (string, error)

SQLIn runs one statement against a named database in this cluster.