The statistics collector process is gone

The process nobody had a graph for

Every PostgreSQL cluster up to and including 14 ran a process that almost nobody monitored and everybody depended on. Backends did not write their own counters anywhere a reader could reach; they sent them as datagrams over a local socket to a collector, the collector accumulated them in its own memory and periodically wrote them into files, and a session that read a statistics view asked the collector to produce a fresh copy of the file for the database it cared about.

That design had properties an operator eventually met. Datagrams could be dropped under load and were simply lost. The files were written often enough that a busy cluster with many tables generated real write traffic from monitoring alone, which is why the received advice was to put the directory on a memory-backed filesystem. And the whole arrangement had exactly one instrument pointed at it, which was a setting naming where the files went.

PostgreSQL 15 removed it. The entry under Monitoring in the PostgreSQL 15 release notes says the cumulative statistics are stored in shared memory, that the data previously travelled by datagram and could only be read after passing through the filesystem, and that there is no longer a separate collector process.

The consequences people usually discuss are about fidelity, and the PostgreSQL 14 end-of-life audit shows what the old path did to a counter reset. This page is about the other half, which is what a 15 server does with the configuration and the directory the old design left behind.

The setting that is not there any more

On 14 the setting exists, and the directory it names has the collector’s working files in it while the server is running.

SHOW stats_temp_directory;
SELECT string_agg(f, ', ' ORDER BY f) AS files_while_running FROM pg_ls_dir('pg_stat_tmp') AS f;

 stats_temp_directory 
----------------------
 pg_stat_tmp
(1 row)

                     files_while_running                      
--------------------------------------------------------------
 db_0.stat, db_13844.stat, global.stat, pgss_query_texts.stat
(1 row)

Per-database files and a global one, exactly as the design implies, plus one file that belongs to the statements extension rather than to core statistics.

On 15, the setting is not merely deprecated. It is unrecognised:

SHOW stats_temp_directory;

ERROR:  unrecognized configuration parameter "stats_temp_directory"

This is the one loud thing on the page, and it is loud in an inconvenient place. An unrecognised parameter in a query is an error in a session. An unrecognised parameter in postgresql.conf is a server that logs a fatal error and does not start. A fleet whose configuration is templated and copied forward, which is most fleets, carries this setting into the new version unless somebody removes it, and the failure lands at the worst possible moment: after the binaries have been swapped, during the window, with the old cluster already shut down.

It is worth being specific about who this catches. Not the team that writes a fresh configuration for the new major. The team whose postgresql.conf has accumulated a decade of settings, several of which nobody currently employed chose, and which is applied by a configuration management tool that has no opinion about which version it is targeting.

What the directory looks like afterwards

The directory itself survives, which is the part that makes the change easy to half-notice.

SELECT string_agg(f, ', ' ORDER BY f) AS files_while_running FROM pg_ls_dir('pg_stat_tmp') AS f;

  files_while_running  
-----------------------
 pgss_query_texts.stat
(1 row)

One file, and it belongs to the statements extension, which keeps its own query-text store on disk and is unaffected by any of this. The core statistics files are gone because core statistics are no longer files.

So a ramdisk mounted on that path is not harmful on 15, and it is not doing what it was mounted to do either. It is now a small memory-backed filesystem holding one extension’s query texts, which is a defensible thing to have and a completely different thing from what the runbook says it is for. If that mount is in your infrastructure code with a comment explaining the write amplification it prevents, the comment stopped being true at the upgrade.

The settings that replaced it

The family of settings around statistics is small enough to read side by side, and the substitution is visible in one query.

SELECT name, setting, context FROM pg_settings WHERE name LIKE 'stats%' ORDER BY name;

On 14:

         name         |   setting   | context 
----------------------+-------------+---------
 stats_temp_directory | pg_stat_tmp | sighup
(1 row)

On 15:

          name           | setting | context 
-------------------------+---------+---------
 stats_fetch_consistency | cache   | user
(1 row)

One setting out, one in, and they are not equivalents. The old one was about where bytes went and was reloadable by the server; the new one is about what a reading transaction sees and can be changed per session, which is what the user context in that output means. It has a page of its own, because the default it ships with surprises people who watch a counter inside a transaction.

Everything else in that family is unchanged, including the master switch that turns counting off entirely. That switch is worth mentioning here only to say that turning it off is now a smaller saving than it used to be: there is no longer a collector process to feed, no datagrams to send and no files to write, so the cost being avoided is the increment itself.

What actually changed for an operator

Four things, in rough order of how likely they are to matter on a given fleet.

• Configuration. The removed setting stops a server from starting. Check it before the window, not during.
• Storage. Whatever the statistics directory was costing in write traffic stops. On a cluster with tens of thousands of tables and partitions that was not a small number, and it is now zero.
• Loss. Counters no longer travel over a socket that can drop them, so the small, unquantifiable undercount that every 14 cluster carried is gone.
• Reading. A statistics view is now assembled from shared memory rather than from a file the collector produced on request, and the reading rules changed with it.

The fourth of those is the one that changes how monitoring behaves rather than how it is configured, and it does not all point in the direction of “faster”. Reading is a different operation now, and on a database with a very large number of relations the cost profile of a full sweep of a statistics view is not the same as it was. That is worth measuring on your own hardware and your own catalog size rather than assuming, because it is the query your collector runs every fifteen seconds forever.

What to grep for before the window

The removed setting is the one that stops a server starting, and it is worth making the check broader than one string, because the same upgrade retires a set of habits rather than a single line.

Start with the configuration files themselves, every one of them: the main file, anything it includes, the automatically managed file the server writes, and whatever template your configuration management renders them from. A setting that has been commented out is harmless; one that is live is not. The check is a grep, it takes a minute, and it is the only item on this page that can turn a routine upgrade into an outage.

Then widen it to the things built around the setting. A mount unit or a filesystem table entry pointing a memory-backed filesystem at the statistics directory will still mount, and will still be doing nothing useful. A monitoring check that asserts the directory exists and has files in it will start failing on a healthy server, because the files it is counting are gone. A backup exclusion listing that directory is now excluding almost nothing.

Finally, look at the documentation. Runbooks that explain why the ramdisk is there, onboarding notes that describe the collector process, and dashboards with a panel labelled after it all become wrong on the same day. None of that breaks anything, and all of it costs the next person time, because a description of a mechanism that no longer exists is worse than no description at all.

The pattern generalises. The expensive part of a removed setting is rarely the setting; it is everything built on the assumption behind it, and none of that is in the release notes because none of it belongs to PostgreSQL.

The thing that did not change

Statistics are still cluster-local and still not part of a backup. They are written out at a clean shutdown and read back at startup, they are discarded after a crash, and they do not travel to a replica or survive a restore. None of that is different on 15, and it is worth saying because the phrase “in shared memory” leads people to assume the data has become more ephemeral than it was. It has not. It was always ephemeral; the ephemerality just used to be visible as a directory you could look at.

The practical form of that is unchanged too. Any comparison expressed as “against the same period last month” needs a stated starting point, because a restart after a crash silently resets it. A monitoring layer that records when counters were last cleared, rather than assuming they never were, is the thing that makes month-over-month statements honest on either version.

What it costs

Nothing, and it removes a cost. There is no setting to enable, no extension, and no restart beyond the upgrade itself. Shared memory usage grows by an amount that scales with the number of objects being tracked, which on an ordinary catalog is unremarkable and on a catalog with hundreds of thousands of partitions is worth knowing exists.

The only migration cost is the configuration line, and it is a two-character edit made at the wrong time if nobody makes it at the right one. The check is one grep across every configuration file you are about to carry forward, and it is the cheapest item on any upgrade list this release produces.