Tomas Vondra

The real cost of random I/O

Thu, 26 Feb 2026 15:00:00 +0200

The random_page_cost was introduced ~25 years ago, and since the very beginning it’s set to 4.0 by default. The storage changed a lot since then, and so did the Postgres code. It’s likely the default does not quite match the reality. But what value should you use instead? Flash storage is much better at handling random I/O, so maybe you should reduce the default? Some places go as far as recommending setting it to 1.0, same as seq_page_cost. Is this intuition right?

The AI inversion

Fri, 20 Feb 2026 12:00:00 +0200

If you attended FOSDEM 2026, you probably noticed discussions on how AI impacts FOSS, mostly in detrimental ways. Two of the three keynotes in Janson mentioned this, and I assume other speakers mentioned the topic too. Moreover, it was a very popular topic in the “hallway track.” I myself chatted about it with multiple people, both from the Postgres community and outside of it. And the experience does not seem great …

Stabilizing Benchmarks

Tue, 06 Jan 2026 12:00:00 +0200

I do a fair amount of benchmarks as part of development, both on my own patches and while reviewing patches by others. That often requires dealing with noise, particularly for small optimizations. Here’s an overview of ways I use to filter out random variations / noise.

Most of the time it’s easy - the benefits are large and obvious. Great! But sometimes we need to care about cases when the changes are small (think less than 5%).

Don't give Postgres too much memory (even on busy systems)

Tue, 23 Dec 2025 12:00:00 +0200

A couple weeks ago I posted about how setting maintenance_work_mem too high may make things slower. Which can be surprising, as the intuition is that memory makes things faster. I got an e-mail about that post, asking if the conclusion would change on a busy system. That’s a really good question, so let’s look at it.

To paraphrase the message I got, it went something like this:

Lower maintenance_work_mem values may split the task into chunks that fit into the CPU cache. Which may end up being faster than with larger chunks.

Qubes OS is pretty great

Tue, 25 Nov 2025 12:00:00 +0200

I’ve been using Qubes OS as my primary OS since version 2, released in 2014. That means I’m using Qubes OS for about a decade. Despite having to deal with a couple issues over the years, I think it’s a great Linux system. I can’t quite imagine switching to a more traditional one. Yet I know very few other Qubes OS users in my developer community, which surprises me. Let me explain why I like Qubes OS, and maybe give it a try.

Wireguard to access a home network

Mon, 17 Nov 2025 12:00:00 +0200

I work remotely from home, and over the years I’ve amassed a bunch of machines related to that (development, testing, benchmarking, …), and other devices you may usually find at home (printer, NAS, …). Occasionally I need remote access, and for a while SSH tunnels were good enough. I decided to simplify and clean this up and use a proper VPN wireguard. This blog post explains the setup I used.

Don't give Postgres too much memory

Fri, 31 Oct 2025 15:00:00 +0200

From time to time I get to investigate issues with some sort of a batch process. It’s getting more and more common that such processes use very high memory limits (maintenance_work_mem and work_mem). I suppose some DBAs follow the logic that “more is better”, not realizing it can hurt the performance quite a bit.

Let me demonstrate this using an example I ran across while testing a fix for parallel builds of GIN indexes. The bug is not particularly interesting or complex, but it required a fairly high value for maintenance_work_mem (the initial report used 20GB).

Tuning AIO in PostgreSQL 18

Wed, 24 Sep 2025 12:00:00 +0200

PostgreSQL 18 was stamped earlier this week, and as usual there’s a lot of improvements. One of the big architectural changes is asynchronous I/O (AIO), allowing asynchronous scheduling of I/O, giving the database more control and better utilizing the storage.

I’m not going to explain how AIO works, or present detailed benchmark results. There have been multiple really good blog posts about that. There’s also a great talk from pgconf.dev 2025 about AIO, and a recent “Talking Postgres” podcast episode with Andres, discussing various aspects of the whole project. I highly suggest reading / watching those.

I want to share a couple suggestions on how to tune the AIO in Postgres 18, and explain some inherent (but not immediately obvious) trade-offs and limitations.

Using JWT to establish a trusted context for RLS

Wed, 27 Aug 2025 12:00:00 +0200

Row-level security (RLS) is a great feature. It allows restricting access to rows by applying filters defined by a policy. It’s a tool useful for cases when the data set can’t be split into separate databases.

Sadly, using RLS may be quite cumbersome. RLS requires some sort of “trusted context” for the RLS policies. The policies need to filter using data the user can’t change. If the filter uses some sort of “tenant ID”, and the user can change it to an arbitrary value, that would break the RLS concept.

This is why solutions like using GUCs are flawed, because the access control for GUC is very limited. The traditional solution is to use roles, which derives the trust from authentication.

It occurred to me it should be possible to build a trusted context on cryptography, independently of authentication. I’ll explain the basic idea, and discuss a couple interesting variations. I’ve also published an experimental extension jwt_context, implementing this using JWT.

I’m interested in all kinds of feedback. Is it a good idea to use JWT this way, as a basis for RLS context? Did I miss some fundamental issue? Are there interesting improvements?

Fun and weirdness with SSDs

Wed, 20 Aug 2025 12:00:00 +0200

When I started working with Postgres (or databases in general) 25 years ago, storage systems looked very different. All storage was “spinning rust” - rotational disks with various interfaces (SATA/SAS/…) and speeds (7.2K/10k/15k/…). The spindle speed was the main performance determining feature, and everyone knew what IOPS and bandwidth to expect from a disk. The general behavior was pretty much the same.

With SSDs it’s more complicated. The interface may be the same, but the hardware “inside” the device can be very different. There’s nothing like the “spindle speed”, a single feature determining fundamental behavior. The flash memory is subject to various limits, but manufacturers may (and do) make different tradeoffs (much more cache, more spare space, etc.). And the hardware changes a lot over time too.

While working on the index prefetching patch, I ran into a couple weird differences between “very similar” queries. And we speculated it might be due to how SSDs handle the different I/O patterns. I did testing on my SSD devices, and there definitely are some very surprising differences in behavior, contradicting (reasonable) expectations. Let’s look at the results, and how it can result in strange query timings.

So why don't we pick the optimal query plan?

Tue, 08 Jul 2025 12:00:00 +0200

Last week I posted about how we often don’t pick the optimal plan. I got asked about difficulties when trying to reproduce my results, so I’ll address that first (I forgot to mention a couple details). I also got questions about how to best spot this issue, and ways to mitigate this. I’ll discuss that too, although I don’t have any great solutions, but I’ll briefly discuss a couple possible planner/executor improvements that might allow handling this better.

How often is the query plan optimal?

Mon, 30 Jun 2025 12:00:00 +0200

The basic promise of a query optimizer is that it picks the “optimal” query plan. But there’s a catch - the plan selection relies on cost estimates, calculated from selectivity estimates and cost of basic resources (I/O, CPU, …). So the question is, how often do we actually pick the “fastest” plan? And the truth is we actually make mistakes quite often.

Consider the following chart, with durations of a simple SELECT query with a range condition. The condition is varied to match different fractions of the table, shown on the x-axis (fraction of pages with matching rows). The plan is forced to use different scan methods using enable_ options, and the dark points mark runs when the scan method “won” even without using the enable_ parameters.

It shows that for selectivities ~1-5% (the x-axis is logarithmic), the planner picks an index scan, but this happens to be a poor choice. It takes up to ~10 seconds, and a simple “dumb” sequential scan would complete the query in ~2 seconds.

Benchmarking is hard, sometimes ...

Thu, 05 Jun 2025 12:00:00 +0200

I do a fair number of benchmarks, not only to validate patches, but also to find interesting (suspicious) stuff to improve. It’s an important part of my development workflow. And it’s fun ;-) But we’re dealing with complex systems (hardware, OS, DB, application), and that brings challenges. Every now and then I run into something that I don’t quite understand.

Consider a read-only pgbench, the simplest workload there is, with a single SELECT doing lookup by PK. If you do this with a small data set on any machine, the expectation is near linear scaling up to the number of cores. It’s not perfect, CPUs have frequency scaling and power management, but it should be close.

Some time ago I tried running this on a big machine with 176 cores (352 threads), using scale 50 (about 750MB, so tiny - it actually fits into L3 on the EPYC 9V33X CPU). And I got the following chart for throughput with different client counts:

This is pretty awful. I still don’t think I entirely understand why this happens, or how to improve the behavior. But let me explain what I know so far, what I think may be happening, and perhaps someone will correct me or have an idea how to fix it.

Advanced Patch Feedback Session (APFS) at pgconf.dev 2025

Thu, 29 May 2025 14:00:00 +0200

The pgconf.dev conference, a revamp of the original PGCon, happened about two weeks ago. It’s the main event for Postgres developers, and one of the things we’re trying is an Advanced Patch Feedback Session (APFS).

We first tried that last year in Vancouver, and then again in Montreal. But I realized many people attending the conference either are not aware of the event at all, or are not sure what it’s about. So let me explain, and share some reflections from this year.

Good time to test io_method (for Postgres 18)

Mon, 12 May 2025 12:00:00 +0200

We’re now in the “feature freeze” phase of Postgres 18 development. That means no new features will get in - only bugfixes and cleanups of already committed changes. The goal is to test and stabilize the code before a release. PG 18 beta1 was released a couple days ago, so it’s a perfect time to do some testing and benchmarking.

One of the fundamental changes in PG 18 is going to be support for asynchronous I/O. And with beta1 out, it’s the right time to run your tests and benchmarks to test this new feature. Both for correctness and regression.

[PATCH IDEA] adaptive execution for `IN` queries

Mon, 28 Apr 2025 12:00:00 +0200

Last week I visited the Malmö PUG to talk about performance cliffs. It’s a really nice meetup - cozy environment, curious audience asking insightful questions. I highly recommend attending or even giving a talk there.

After the meetup I realized it’s been a while since I posted about some patch idea, and the performance cliff talk has a couple good candidates. Some might be a bit too hard for the first patch, for example improving the JIT costing. But improving the very first example about queries with IN clauses seems feasible. It’s quite well defined and isolated.

15 years of Prague PostgreSQL Developer Day

Wed, 26 Mar 2025 12:00:00 +0200

It’s been a couple weeks since P2D2 (Prague PostgreSQL Developer Day) 2025. We’ve been busy with the various tiny tasks that need to happen after the conference - processing feedback, paying invoices, and so on. But it’s also a good opportunity to look back - I realized this was the 15th year of the event I’ve helped to organize, so let me share some of that experience.

Performance archaeology: OLAP

Tue, 03 Dec 2024 14:00:00 +0200

A couple days ago I wrote about performance improvements on OLTP workloads since Postgres 8.0, released 20 years ago. And I promised to share a similar analysis about analytical workloads in a follow-up post. So here we go ;-) Let me show you some numbers from a TPC-H benchmark, with some basic commentary and thoughts about the future.

Performance archaeology: OLTP

Tue, 26 Nov 2024 16:00:00 +0200

The Postgres open source project is nearly 30 years old, I personally started using it about 20 years ago. And I’ve been contributing code for at least 10 years. But even with all that experience I find it really difficult to make judgments about how the performance changed over the years. Did it improve? And by how much? I decided to do some benchmarks to answer this question.

Tuning the glibc memory allocator (for Postgres)

Mon, 14 Oct 2024 12:00:00 +0200

If you’ve done any Postgres development in C, you’re probably aware of the concept of memory contexts. The primary purpose of memory contexts is to absolve the developers of having to track every single piece of memory they allocated. But it’s about performance too, because memory contexts cache the memory to save on malloc/free calls. But malloc gets the memory from another allocator in libc, and each libc has its own thing. The glibc allocator has some concurrency bottlenecks (which I learned the hard way), but it’s possible to tune that.

[PATCH IDEA] parallel pgbench -i

Tue, 01 Oct 2024 12:00:00 +0200

There are multiple tools to run benchmarks on Postgres, but pgbench is probably the most widely used one. The workload is very simple and perhaps a bit synthetic, but almost everyone is familiar with it and it’s a very convenient way to do quick tests and assessments. It was improved in various ways (e.g. to do partitioning), but the initial data load is still serial - only a single process does the COPY. Which annoys me - it may take a lot of time before I can start with the benchmarks itself.

This week’s “first patch” idea is to extend pgbench -i to allow the data load to happen in parallel, with multiple clients generating and sending the data.

Playing with BOLT and Postgres

Wed, 25 Sep 2024 12:00:00 +0200

A couple days ago I had a bit of free time in the evening, and I was bored, so I decided to play with BOLT a little bit. No, not the dog from a Disney movie, the BOLT tool from LLVM project, aimed at optimizing binaries. It took me a while to get it working, but the results are unexpectedly good, in some cases up to 40%. So let me share my notes and benchmark results, and maybe there’s something we can learn from it. We’ll start by going through a couple rabbit holes first, though.

[PATCH IDEA] amcheck support for BRIN indexes

Tue, 17 Sep 2024 12:00:00 +0200

Time for yet another “first patch” idea post ;-) This time it’s about BRIN indexes. Postgres has a contrib module called amcheck, meant to check logical consistency of objects (tables and indexes). At the moment the module supports heap relations (i.e. tables) and B-Tree indexes (by far the most commonly used index type). There is a patch adding support for GiST and GIN indexes, and the idea is to also allow checking BRIN indexes.

Writing a good talk proposal

Tue, 10 Sep 2024 12:00:00 +0200

I’ve submitted a lot of talk proposals to a lot of Postgres conferences over the years. Some got accepted, many more were not. And I’ve been on the other side of this process too, as a member of the CfP committee responsible for selecting talks. So let me give you a couple suggestions on how to write a good talk proposal.

[PATCH IDEA] Statistics for the file descriptor cache

Tue, 03 Sep 2024 12:00:00 +0200

Let me present another “first patch” idea, related to a runtime stats on access to files storing data. Having this kind of information would be very valuable on instances with many files (which can happen for many reasons).

This is a very different area than the patch idea, which was about an extension. The runtime stats are at the core of the system, and so is the interaction with the file systems. But it’s still fairly isolated, and thus suitable for new contributors.

Office hours experiment

Sat, 31 Aug 2024 12:00:00 +0200

I’ve decided to experiment a little bit and do regular “office hours.” I’ll be available to chat about almost anything related to Postgres. It might be a technical discussion about a patch you’re working on, or a topic about the community etc.

This is not an entirely new thing. I’ve been telling people to just ping me if they want to discuss something off-list, or have a call and chat about it. I did have a couple such calls, and it was nice - faster than discussing that by email, maybe a bit closer to the ad hoc watercooler talk or a hallway track. So I’m mostly just announcing this more widely, with a couple simple rules.

[PATCH IDEA] Using COPY for postgres_fdw INSERT batching

Tue, 27 Aug 2024 12:00:00 +0200

In an earlier post I mentioned I plan to share a couple patch ideas, suitable for new contributors. This is the first one, about using COPY protocol for postgres_fdw batching. This would replace the current implementation, based on prepared statements. Let me share a couple thoughts on the motivation and how it might be implemented.

Importing Postgres mailing list archives

Fri, 23 Aug 2024 12:00:00 +0200

A couple weeks ago I needed to move my mailing list communication to a different mailbox. That sounds straightforward - go to the community account and resubscribe to all the lists with the new address, and then import a bit of history from the archives so that the client can show threads, search etc.

The first part worked like a charm, but importing the archives turned out to be a bit tricky, and I ran into a bunch of non-obvious issues. So here’s how I made that work in the end.

How to pick the first patch?

Tue, 20 Aug 2024 12:00:00 +0200

Picking the topic for your first patch in any project is hard, and Postgres is no exception. Limited developer experience with important parts of the code make it difficult to judge feasibility/complexity of a feature idea. And if you’re not an experienced user, it may not be very obvious if a feature is beneficial. Let me share a couple simple suggestions on how to find a good topic for the first patch.

Will Postgres development rely on mailing lists forever?

Tue, 13 Aug 2024 12:00:00 +0200

Postgres is pretty old. The open source project started in 1996, so close to 30 years ago. And since then, Postgres has become one of the most successful and popular databases. But it also means a lot of the development process reflects how things were done back then. The reliance on mailing lists is a good example of this heritage. Let’s talk about if / how this might change.

The state of the Postgres community

Sat, 03 Aug 2024 12:00:00 +0200

About a month ago I presented a keynote at Swiss PGDay 2024 about the state of the Postgres community. My talk included a couple charts illustrating the evolution and current state of various parts of the community - what works fine and what challenges will require more attention.

Judging by the feedback, those charts are interesting and reveal things that are surprising or at least not entirely expected. So let me share them, with a bit of additional commentary.