How to Manage a PostgreSQL Commitfest

I've been interacting with open-source software communities professionally for most of my career. My first introduction to PostgreSQL was around 2014, when it seemed a good choice for a web stack I was working on. In 2017, I switched my career focus from open-source web servers to open-source databases, and since 2020 I've been developing with and contributing to PostgreSQL full-time.

Earlier this year, I managed the July commitfest, the first of five community review events for the future PostgreSQL 16. While there is some information about how to manage a commitfest (CF), it's starting to get out of date, and most of the official documentation focuses on the mechanics rather than the people. With the September CF coming soon, I wanted to share my experience as a new Commitfest Manager (CFM) in more detail, along with some advice to other new CFMs.

Managing a PostgreSQL Commitfest: The Basics

Being a CFM entails a lot of administrative work to keep community reviews flowing smoothly. In a previous blog post, I wrote a list of recommendations for those aiming to be CFMs (including pointers to the Commitfest App, the cfbot CI interface, and the pgsql-hackers mailing list) and gave a quick overview of the tasks that come with the role.

This post builds on that one, so take a look if you haven’t already. Still, I will repeat this one piece of advice: potential CFMs should already feel comfortable interacting with the PostgreSQL development community. Your job is primarily communication, so experience with the mailing list is important.

Like the last post, the contents of this one are my personal opinion only. If you’d like official rules and positions from the community, see the PostgreSQL Code of Conduct.

A Possible Commitfest Timeline

From your perspective as a CFM, let's run through a timeline of events. A CF runs for one month. Your duties will start roughly a week prior to that, and end a couple of days afterward.

This timeline mirrors my experience, but you don't have to do it the same way. The great thing about a rotating volunteer position is that everyone can bring new ideas and their own personal take.

Prerequisite: Volunteer

You should volunteer to be CFM the month before the CF you'd like to manage. Send an email to -hackers; ideally, someone will come along soon and tell you if there's already been a volunteer (in which case you may want to join forces as an assistant) or if you've got the job. It's a busy mailing list, and code discussions can sometimes drown out administrative work, so don't be afraid to send a follow-up in a few days if you haven't received a response.

Once someone has confirmed that you'll be managing, request administrator access to the CF App, by sending an email to the -www list.

Preparation: Initial sweeps

With a week to go, send a reminder to the list that the CF will begin soon and that people should register their patches. (You'll be following up on this reminder shortly.)

Now, we'll try to catch patches that have fallen through the cracks. First-time contributors who send their patch sets don't always know that commitfests exist. It's usually the whole community's job to tell them this, but during busy periods (especially at the end of a CF!), this doesn't always happen. Your first job is to sweep the -hackers mailing list for any patches that were posted after the prior CF started and that do not have a corresponding entry in the CF app.

This will be easiest if your mail client has a robust search function. You'll want to restrict by date (the last two months, in most cases, but note that the first CF of a new PostgreSQL development cycle has a gap of four months to search!) and search for messages with attachments. Send a quick email to the authors if there is no corresponding CF entry, and ask them if they'd like to register.

Sometimes the answer will be "no." That's okay. Sometimes you won't receive an answer. That's okay, too. Your job during this phase is to ensure that no one gets left behind simply because they don't know what to do. (If this seems to you like something we should be automating, I agree.)

Once that's done, we can do our first status sweep.

During a CF, the various threads on the list begin a back-and-forth between reviewers and authors. If an entry is in the Needs Review state, the author is looking for someone to provide feedback of some sort. If the entry is Waiting on Author, the reviewers (or the community) need some action to be taken by the author before the patch set can progress. Once reviewers are satisfied, they mark the entry Ready for Committer to signal to someone with a commit bit that they should take a look. (This may—and often does—kick off another round of reviews and changes.)

Since this is all tracked manually, it can quickly fall out of date. Now's our chance to fix things up a bit before the next CF starts (and simultaneously familiarize ourselves with some of the patches). Go through the CF entries and look for patches that might have misaligned statuses. For example, a Waiting on Author thread whose last email had an attachment from the author might need to be switched back to Needs Review. Or a Needs Review thread with several outstanding requests and no follow-up by the author might need to be changed to Waiting.

There are a lot of patches, so you may want to reduce the scope for this first sweep. A misaligned status for an active thread, where there's currently a lively conversation, will probably not be a problem for anyone in practice. But for a quiet thread, starting the next CF in Waiting on Author might mean the thread continues to stay quiet, which is particularly problematic if the author thought they had already responded to the provided feedback and is now waiting in vain. So I focused on entries that appeared to be "stale"—that hadn't received any feedback or author updates since the last CF.

(This situation also contains a good lesson for reviewers: don't assume that every thread in the Waiting on Author status is unreviewable! You may be able to provide helpful feedback even when a patch no longer applies, and in the best case, you may find that the author forgot to update the patch status.)

cfbot can also help us here. A patch set that hasn't been applied cleanly to the HEAD of the PostgreSQL repository in weeks is probably not going to be very easy to review (assuming any reviewers want to try it in the first place). And a Ready for Committer entry that fails to apply when a committer comes along to try it can lead to a frustrating return to Waiting on Author. A quick email on the thread asking the author to rebase can help keep things moving smoothly. (Be judicious with these, especially if the author has already shown that they're using cfbot. Otherwise, rebase requests can become lots of noise with little signal.)

Beginning the commitfest

The day before the CF starts, send one last reminder to the list so that stragglers can get their patches registered. Then we wait for the fun to begin.

Once the new month has begun everywhere on Earth, kick off the CF by marking it In Progress via the administration link at the top of the CF App. This will close it to any new entries, and you'll need to ensure that the next CF is marked Open for any new patches people send in during the month.

Send a quick note to the -hackers list stating that the CF has started. Some CFMs like to keep a weekly drumbeat with the commit statistics; if you want to do that, you can take this opportunity to record the starting numbers for posterity.

Finally, use the CF App to send an email to all authors

   [Send private mail > All authors (open patches)]

reminding them that they have patches registered for the current commitfest and that they should plan to review other patches of similar complexity to the ones they've submitted. Ideally, this helps remind people that they're expected to donate back to the community while their entries are receiving reviews from others.

Each week: Status sweep

Starting a week after the opening day and roughly every week after that, make a quick check for incorrect statuses in the same way you did before. Roughly halfway through, keep an eye on patches that seem to be getting stale; this will help us during the final triage. (If you see anything that could help unblock the patch, mention it on the thread.)

You can also send a regular drumbeat (We've committed X patches! Readied Y others!) as a bit of cheerleading for the process. People are people, after all, and it's nice to be reminded of progress, especially if the to-do list is long.

Final week: (Optional) triage

The CF is beginning to wind down now. Time to start deciding which patches need to continue as-is to the next CF and which need to be removed for now.

This is easily the most challenging part, and since contributors often (rightly!) have a personal investment in their work, emotions can occasionally run high. If you're brand new, this is a good thing to ask for help from your assistant CFM. (And if you don't have an assistant or don't feel comfortable performing this level of thread analysis, remember that it's optional. Not every manager does this.)

Here are the various closure states you might use as of this writing:

• Moved to next CF. This means that the community plans to continue working on this patch. It's by far the most common closure state.
• Returned with Feedback (RwF). This is currently the catch-all, covering the bases from "needs to be rebased" to "changes requested" to the extremely vague "the community can't provide a review for this right now." It implies that the author needs to take some action, then reopen the thread for review in a future CF once they are ready. We also apply this at the end of a CF if a patch has been Waiting on Author for at least two weeks without any author response.
• Rejected. This is uncommon. It means that the community has decided that the idea or feature presented in the entry will not be part of PostgreSQL, and we don't think any amount of code work on the patch set will change that. It is a "final" state (at least, until something fundamental changes in the project). In general, this is not something you decide on your own as a CFM; it's something that someone else—usually a committer— will propose and build a consensus on.

Additionally, there are two more "final" states that you should be aware of:

• Committed. Hooray, a patch set got in! Usually, you won't set this yourself: a committer will mark the entry as Committed after they push to the repo.
• Withdrawn. This means that an author has voluntarily removed the patch from the review pool. You should not use this status as a CFM unless, of course, you are the author or the author has reached out and asked you to withdraw the patch for them.

Since the CF isn't over, we're not actually going to use any of these yet. Instead, we'll survey the list and start taking notes on which patches we plan to move where. It's an extensive list, so I recommend spending the majority of your time looking over the older, staler entries. Triage emails sent by previous CFMs can additionally help you find any historical discussions or calls for closure.

Here are some of the special cases I looked for when I was CFM:

• Patches that have been in Needs Review for many (four or more) CFs without any reviewer feedback. These may be difficult to review for some reason, or perhaps the community doesn't have the bandwidth or desire to take on the patch set right now. These are sometimes good candidates for return.

(Returned with Feedback is a bit of a misnomer for these since there hasn't actually been any feedback. There has been a discussion recently about introducing a "needs more interest" close state, but for now, RwF is what we have.)

• Patches that have been stuck in Ready for Committer for months (or several separate times) without actually being committed. Try to find what's holding things up. Is the entry really Waiting on Author? Have committers expressed concern over the changes (or suggested Rejection)? Does the author need help to keep the set rebased?

For any patches you think should be returned, notify the entire list in a triage email and send a copy (CC) to the authors. This will let people discuss and raise objections before you make accidental busy work for them next week.

Finally, the day before the CF ends, send a quick reminder that the event is almost over, and you'll be moving entries when it closes. Then, we wait.

The day after: Closing the commitfest

It's no longer CF month anywhere on Earth! Time to close it all out. Here's where our triage work pays off.

Close every open patch remaining in the CF with an appropriate status. Here are the rough guidelines I used when I managed in July, based on community suggestions:

• If an entry has recent back-and-forth from the author and reviewers, it gets moved to the next CF without further ado.
• The same goes for entries that have not yet received any review and are not on your triage list. On to the next CF they go.
• If an entry has been Waiting on Author for two weeks or more, and the author has not yet responded, it gets Returned with Feedback. Send a short email on the thread to let the author know how to reopen the entry once they're ready.
• Use your triage list and follow-up discussions to close out most of the remainder as planned. Send explanatory emails as you go: authors should know their next steps.
• You'll probably have some leftovers which you should move to the next CF unless there's already been another proposal. I tend to view a CFM's role as a facilitator more than a gatekeeper: CFMs should not be personally gating patches. If you have concerns, you can voice them on the thread and let the community decide—but move them ahead for now since we need to close the CF.

With the number of things to remember here, mistakes will be made. Don't let that paralyze you, as long as you think action will be helpful and you're ready to undo that action if it turns out otherwise. Especially when fixing a mistake is as simple as changing a patch status.

After every patch is closed, you can mark the CF as Closed in the administration interface. (Don't do this while patches are open because they'll interfere with the cfbot's continuous integration systems.) Now's a good chance to send the final tally of commit statistics and congratulate the community on a well-done job.

And congratulations to you, too! You made it.

General Advice

The following applies to open-source interactions in general, but hopefully it serves you well.

It's easy to fall into the trap of placing people on a line from "newbie" to "expert," but that's not really how people work. A hypothetical new contributor may be an expert in advanced database optimization and still not know much about contributing or interacting with an open-source community. Everyone has something to learn; try to facilitate that learning as much as possible.

I know that a significant shift (or shock) for many developers is the "mailing list culture," particularly the expected turnaround time between sending a message and receiving a response. You should give people a week (or two!) of leeway, and here's why: on a mailing list, you will interact with people across time zones, languages, cultures, skill sets, and backgrounds.

Some people are paid to do this, but some do this purely in volunteer time, on their nights and weekends. Some people are practiced at reading and writing in English, and some are awake at the same time you are, but many more are not. Don't punish people for not being able to turn around a response within a day or even five. Give these volunteers the time they need to keep volunteering.

Finally, find balance! I mentioned that your role is not that of a gatekeeper but of a facilitator. In the face of an ever-growing list of to-dos, and as Postgres gains popularity, there's sometimes a need to close out patches that the community does not realistically have the resources to adopt. And that needs to be balanced with empathy and real gratitude to the people spending their free time improving the ecosystem. You can help find a path forward.

End Remarks

A year ago, my teammate Aleks pointed out that, according to our 2021 community survey, only 15 percent of Postgres users had contributed back to the project. As of our 2022 survey, this number hasn't changed much—only 17 percent of surveyed users have contributed at least once—and considering free-form responses like the following, it's clear that we have work to do:

“CommitFest App and workflow is ... awkward.”

“The mailing lists are not very user-friendly. It's hard to see open bugs, and it's difficult to follow discussions.”

“I wish there were a way to communicate our needs as users to the developers, a way that *isn't* the pg-hackers mailing list or cornering someone at a conference.”

“I'm not really aware of any ‘real community,’ I usually find my solutions from StackOverflow or blogs.”

This article is mostly a description of how things are, rather than suggestions for improvement. But the more visible the current process is, hopefully, the easier it will be for newer community members to be able to jump in, participate, and feel comfortable making those suggestions for change.

And, should you decide to volunteer as CFM, thank you! Best of luck as you help welcome those new voices to the project, and I’ll see you on the list.