Double buffering in PostgreSQL

Double buffering is the ordinary situation where the same 8 kB page is held twice in memory: once in PostgreSQL’s own shared buffer pool and again in the operating system’s page cache. PostgreSQL reads and writes through normal file operations rather than bypassing the kernel, so every block it fetches passes through the kernel’s cache and stays there. The duplication is not a misconfiguration. It is the consequence of a design decision to let the operating system manage the larger cache.

What it means for the settings you choose

The first consequence is that shared_buffers is not the server’s memory budget. It is the size of the pool PostgreSQL manages directly, and the rest of the machine’s free memory is still working for the database, just under someone else’s replacement policy. This is why the old advice to give PostgreSQL most of the machine is wrong on most systems: a very large pool takes memory away from a cache that was also serving those pages, and gains you only the difference between two policies.

The second is that effective_cache_size is a planner input describing the total, both layers together, and it allocates nothing at all. Setting it does not consume memory and does not change what is cached. It changes how cheap the planner believes an index scan will be, which is a plan-shape decision rather than a memory one, and the two settings are confused for each other constantly.

The third is what it does to measurement. A block that misses in shared buffers and hits in the page cache is counted as a read by PostgreSQL and takes microseconds. From inside SQL the two cases are indistinguishable, which is why the cache hit ratio cannot answer the question people ask of it. Timing is the only signal that separates them, and it has to be switched on.

Looking at the half the server can see

The buffer pool can be inspected directly; the layer below it cannot.

CREATE EXTENSION pg_buffercache;
SELECT current_setting('shared_buffers') AS shared_buffers,
       count(*) FILTER (WHERE relfilenode IS NOT NULL) AS buffers_in_use,
       count(*) FILTER (WHERE relfilenode IS NULL) AS buffers_free
FROM pg_buffercache;

CREATE EXTENSION
 shared_buffers | buffers_in_use | buffers_free 
----------------+----------------+--------------
 128MB          |            922 |        15462
(1 row)

A default 128 MB pool on 14.24 with roughly a twentieth of it occupied, because the server had barely been used. This is the complete picture of one layer and none of the other: there is no SQL interface to the kernel’s cache, and the same page counted as in use above may also be sitting in the page cache, or may not. Any accounting of database memory built only from this view is describing half a system.

From PostgreSQL 16, pg_stat_io at least tells you which process type is doing the reading and in what context, and with timing enabled the durations distinguish a page-cache hit from a storage read well enough to act on. The page on the I/O statistics view in PostgreSQL 16 covers what it costs to turn that on.

Sizing it with the layers in mind

Deciding how much memory to give the pool, and what the newer majors changed about how the I/O underneath it is accounted for, is covered in what PostgreSQL 18 changed in monitoring, which is also where the asynchronous read path and its effect on these numbers is described.