Also, it’s our team’s favorite engine to translate both foreign SQL and foreign applications for PostgreSQL! We believe in transitions, not migrations - a gradual shift from the old database engine to the new, specifically to reduce risk and optimize results. Using a project like this makes a huge difference in achieving both these goals.
It’s been a little bit since the latest major version was released, but we’re still excited about what’s new and wanted to share.
Version 5.0 (release notes) came out on November 25, 2025 and there’s already been a minor version released since then (5.1 release notes) on December 18, 2025. These editions of the project were the direct result of massive efforts from the IvorySQL team to provide a lot of quality feature improvements as well as implement PostgreSQL 18 compatibility.
IvorySQL 5 brought a lot of necessary changes with it, including:
In particular, we really enjoy a lot of the functionality upgrades.
For example, IvorySQL now has better NULL handling, where NULL is now treated like an empty string in compatible mode (matching Oracle behavior) to avoid bugs during migration. So, when you have situations such as SELECT CONCAT('a' || NULL), IvorySQL will now return 'a' instead of following PostgreSQL’s behavior of returning NULL.
One enhancement we really enjoy is that you can now nest functions and procedures. Functions are able to be embedded inside other functions (like Oracle packages, but simpler, using only private methods). This lets you organize complex logic in one place.
Along the same lines, support was also added for DO [ LANGUAGE lang_name ] code [USING IN | OUT | IN OUT, ...].
Currently, we’re ingesting applications that range from only a few packages, procedures, and functions (around 10-50 objects) to massive data loads that include around 50,000 objects (10,000+ procedures and hundreds of packages).
While there are still many features that will need to be added over time, it considerably reduces the effort taken to complete the transition from Oracle to PostgreSQL successfully.
If you’re thinking about testing it for yourself, there are a plethora of options to get started (including building it from source, using containers, etc.)… and even a WASM build! This in particular allows IvorySQL to be run directly in the browser without having to fully install it locally; it’s very convenient to look at syntax support and take the first step into the world of PostgreSQL.
You can deploy the IvorySQL-WASM project locally in just a few steps, following the blog post IvorySQL published on the subject.
Or, you can try out the hosted WASM on the IvorySQL website.
Hoping to learn more about how to use IvorySQL yourself? Keep an eye out for recorded webinars on their website (coming soon!), or check out recordings from the HOW2025 conference in Jinan, China in the meantime.
The next conference in Jinan is already scheduled, from April 26th to the 28th! (By the way, the Call for Proposals is open until February 27th, 2026.)
Our team is happy to employ contributors to the IvorySQL project. Some of our team members (Cédric Villemain, Yasir Hussain Shah) have been recognized in the release notes for 5.0 and 5.1 - and we have more additions planned. For the next release, we have planned some exciting developments, including:
ENABLE/DISABLE constraint syntaxUTL_FILE package additionCREATE TRIGGER body (without needing to create a function beforehand)(+)We’re looking forward to seeing the result of ongoing community work for the next release as well. Many great features are actively being added; for example, recently a new contributor to the project, Rophy Tsai, added the DBMS_OUTPUT and DBMS_UTILITY packages: something widely used across Oracle-based SQL codebases. It was recently merged to IvorySQL’s main branch, and will likely be included in the next minor or major release.
For those looking to transition to or explore PostgreSQL without extensive modifications to their existing database logic and applications, IvorySQL (specially v5.x) offers a practical solution. A direct migration to PostgreSQL could, in some cases, require significant changes to both database objects and application code. IvorySQL helps bridge this gap, allowing a smoother adoption of PostgreSQL with greater compatibility.
If you find the subject of particle physics interesting, you may want to visit anyways! They offer free access to many activities that run from Tuesday to Sunday; you can view the full programme here.
Here, you’ll be able to attend a single track of seven English-language sessions, with a social gathering afterwards to enjoy CERN while continuing to connect with the rest of the attendees.
This year, there’ll be:
The first talk of the day is of particular note…
The CERN Tape Archive (CTA) stores over one exabyte of scientific data. To orchestrate storage operations (archival) and access operations (retrieval), the CTA Scheduler coordinates concurrent data movements across hundreds of tape servers, relying on a Scheduler Database (Scheduler DB) to manage the metadata of the in-flight requests. The existing objectstore-based design of the CTA Scheduler DB is a complex transactional management system. This talk presents the development of a new PostgreSQL-based backend for the CTA Scheduler as an off-the-shelf solution which simplifies implementation and is expected to significantly reduce future development and operational costs. We describe the implementation of all main CTA workflows and explain how PostgreSQL addresses the limitations of the objectstore-based system, providing the foundation for the tenfold increase in data throughput expected during LHC Run 4.
In a world where ever larger amounts of digital information must be stored, learning more about how CERN manages over one exabyte of scientific data is sure to be an interesting experience.
Geneva is home to many international organizations across the public, private, and scientific sectors. If you’d like to explore the topic of PostgreSQL in more depth through engaging in discussion or attending sessions, it’s a fun location to meet and learn. Thinking of coming by? You can register until February 1st.
Last year’s session recordings can be viewed by visiting the 2025 schedule and selecting the paperclip symbol next to the talk you’re interested in.
Stop by and see us in the catering area; we’re proud to be sponsoring the event again this year and will have a table or booth where you can visit. We’d love to talk about what PostgreSQL and open-source innovation, development, & whatever questions you have.
]]>That particular atmospheric event followed major heatwaves and broken records across the U.S., Europe, and Asia in the past few years. Just this last week at the time of this article, there has been historic flooding with profound damage in Washington state (U.S.)… more than nine million acre-feet of water dumped on the state over ten days… with the torrential downpour predicted to continue throughout the West Coast the rest of the year.
I don’t know if the relevancy of Ibiza’s floods in particular played a part in the way my short talk at the conference was received, but I am still grateful the topic at hand was not only heard, but listened to - even thought about and discussed. I must admit I was not expecting that.
I am part of those who believe that global environmental challenges are about to completely transform our societies on every level as resources get more limited and redistributed. I am the one asking around if anyone else wonders “what good will my comfortable salary do me if potatoes can’t grow anymore?”. Maslow clearly illustrated that it is less necessary to be rich than fed. Yet, as we all know, the environment is still not priority one, even after all this time and all the lessons we’ve experienced.
So, for me it made sense to discuss the topic at a PostgreSQL conference, because it should simply be discussed everywhere. It is a topic that is relevant to each and every one of us, because it involves the Earth itself that we reside on and take for granted. Yet, was I confident about finding a receptive audience? Nope!
I felt slightly uncomfortable – I usually do as a non-tech person at a technical event… – for taking the audience into a space they were not necessarily there for. I was hoping I would not bore, or bother, or even annoy. But after only the first few slides I saw that people were taking pictures of the data I was sharing, some were nodding, some were seeing those figures for the first time and raised an eyebrow. And when I linked it all with what we do daily and the impact of the digital pollution we produce, instead of facing disinterest or disapproval I got “why not!” looks in the room.
According to Greenpeace, digital pollution refers to all forms of pollution caused by the IT sector: greenhouse gas emissions, chemical contamination, erosion of biodiversity, and production of electronic waste.
Yes, there are some innovative solutions being designed to handle some of this digital pollution. For example, some situations are ideal for creating data centers that are able to use the heat they produce in order to heat nearby offices or homes. Projects such as Amazon’s Tallaght project is “the first large-scale district heating network of its kind in Ireland”, helping to “significantly reduce emissions in the area by almost 1,500 tonnes of CO2/year, establishing Tallaght as a leader in local energy action” (source).
However, the growth in the number of users equipped with at least one connected terminal (especially in developing countries), the increase in the ratio of the number of connected terminals per individual (from 2.1 in 2015 to 3.3 in 2020 on average worldwide), the increase in video traffic coupled with the increasing share of HD and UHD quality images and the shift of usage to consumption on demand (streaming, VOD, cloud gaming) have resulted in an explosion of traffic on networks (more than 25% per year) and in data centers (more than 35% per year). This growth is occurring at a rate that surpasses energy efficiency gains in equipment, networks and data centers. These traffic forecasts are also regularly revised upwards.
Although the number of “standard” devices (excluding connected objects) has increased significantly between 2000 and 2015, it stabilises between 2015 and 2025 because the market is saturated.
Although the mass per user decreases slightly, from 63 kg CO2 emmissions to 58 kg between 2010 and 2025, the total mass (user equipment, networks, data centres) is multiplied by more than 2.5 in 15 years: it increases from 128 million tonnes in 2010 to 317 million tonnes in 2025.
This explains the stress on raw materials, especially “conflict minerals” and other rare earths.
Performance optimization for databases doesn’t just result in improved costs and speed. When we work on improving a query, we save that little bit of extra energy that would have been used to read/write, that extra disk movement that was always unnecessary. Other essential responsible practices that in a clever way allow you to better harness the power of technology results not just in a more technically efficient result but also - without realizing or thinking about it - doing some good for the planet.
For example, some actions we commonly consider for our own good that have an equally beneficial result for our planet include:
We need to think about digital sobriety, thoughtful coding practices and considerate energy consumption as being the key to greener coding and integrate that parameter in the way we work every day and in the way we build, manage and develop database solutions in order to minimize our environmental footprints as much as possible. These kinds of actions can generate significant cost savings for our own companies and for our customers, which can then be reinvested in R&D projects to increase energy efficiency. It’s a virtuous circle.
A closing thought to this section: AI in particular drives far higher water usage, emissions, and e-waste (training AI models is one of the most resource-intensive computing tasks on the planet!). In a world where it has become common to integrate AI into every corner of applications and day-to-day coding practices, being deliberate in how such technologies are used to ensure they are truly the best solution for the problem at hand has a huge impact. Is AI really the most appropriate solution for that issue you’ve been thinking about? Or are you considering its usage merely because it’s easier to implement than something more technically involved?
And that’s an interesting twist to consider regarding our day-to-day. We sometimes complain about needing to know there’s meaning to what we do; well here’s some food for thought to fuel your complaints!
A brief note from our Marketing team here at Data Bene: our team of experienced PostgreSQL engineers can help with improving your own energy efficiency, database optimizations, and thoughtful coding practices that will result in direct benefits to you: speedier database operations, decreased resource usage, lessened costs, and a greener planet. Curious about what we offer? Get in touch - we’d love to chat.
]]>The event lasted two days, featuring 90 exhibitors, 130 sessions, 150 speakers, and over 4,000 participants – a truly large-scale conference held at the Cité des Sciences et de l’Industrie in Paris.
This particular venue was smaller than last year’s venue (Le Palais de Congrés) and the rooms we went to were all full as a result of being very small (about 20 or 30 seats or so at most).
Presentations were given in both English and French. Interestingly, there were no “silent rooms” this year (where headphones are provided to each attendee). Not everyone enjoyed that format last year, but it was a useful one for following two talks, or switching between them depending on the content or questions.
Two of our team members were in attendance and had the opportunity to explore various exhibitors and event rooms spread across three floors. The talks lasted 20 minutes. While too short to delve into details, this format was excellent for discovering new technologies and piquing our interest at a glance.
There were six tracks that talks were categorized by:
We found the topic of open source solutions within the public sector to be the most interesting. In particular, it was easy to see our reliance as a global society on the big five tech companies (GAFAM: Google, Apple, Facebook, Amazon, and Microsoft) has grown significantly in the past few years. Open source software is a direct solution for protecting our collective right to privacy in the digital age, which is exactly why conferences such as this are so important for the discovery of OSS alternatives and innovation that leads to further development within this sector.
Attendance was particularly high from the very first day. We were delighted to have the opportunity to interact in person and meet our partners, especially OW2, which also organizes an annual Open Source event in June. (The call for presentations is open until February 14, 2026 – see the OW2Con’26 call for proposals here.)
Since the event was entirely focused on open source technologies, we were able to discuss with numerous participants topics such as PostgreSQL support, along with the challenges and organizational impacts for companies wishing to innovate and adopt PostgreSQL, against the backdrop of market demand to break free from proprietary software licensing constraints.
We also had the pleasure of meeting key players in open source hardware innovation, which resonates with our own R&D on RISC-V processors.
Many free software and open source projects were represented. Some examples include Nextcloud (a self-hosted cloud collaboration platform that we personally use for hosting here at Data Bene) and OpenTalk, a video-conferencing solution that is GDPR-compliant, operating within German data centers.
The event was successful and well-organized. The only thing that would have improved the experience would have been longer presentations to explore the various topics discussed in more depth. If you want to discover new open source projects, note that this event is also a great opportunity to freely exchange ideas on these topics.
The video replays for 2025 have not yet been published, but past conference recordings can be found on the official website, here.
Overall, we thoroughly enjoyed the event and hope to attend next year!
]]>This was the first time that the Data Bene team attended the event, and we’re happy to share that it was a very positive experience.
As its name suggests, this conference takes place on the Mediterranean island of Ibiza. For those who are less familiar, the Official Tourism Site provides a nice overview that you may refer to. While it should go without saying, this UNESCO World Heritage Site has incredible offerings outside of the conference itself. Combined with a potentially long-haul flight, it’s strongly recommended to plan an extra day or two (or more) to explore the island and enjoy the local experience.
The event itself is hosted at the Palacio de Congresos de Ibiza; a two-story conference center with multiple rooms and catering. Though located outside of the capital city, the venue can be reached by car in good timing (~15 minutes). To add, the venue is situated within walking distance of numerous hotels, restaurants, and a beach! We happened to have stayed at Hotel Tres Torres, which we can recommend, though admittedly it’s difficult to make a wrong decision, especially following a sprinkle of background research.
Briefly, on the topic of extracurriculars, our go-to recommendation would be to visit the capital city, Ibiza. There you will be able to find a surprisingly large amount of shopping, restaurants, and sightseeing. Adding to that a walk through the historic neighborhood and you’ve got quite the memorable experience.
This year’s PGIBZ welcomed about 30-40 attendees from across the globe, including China and the United States. And while there are certainly two sides to the coin, I consider this group size as more of a positive. Indeed, during both the presentation Q/A sessions, as well as the coffee breaks, I felt interactions were more engaging overall. This was especially true when meeting and reconnecting with members of both more modest and global enterprise-scaled companies.
The first of the Data Bene speakers, and indeed the keynote of the first day, was Catherine Bouxin. Her presentation addressed the environmental impact of technology, and reflected the philosophy that important topics are not always the ones being discussed, and often aren’t considered a concern at all! Indeed, Catherine framed this conversation primarily as a reminder of the responsibility we as PostgreSQL community members share. There have been many exciting inventions of late, some with societal-changing implications, but we can not ignore the resource intensity that many of them require.
We stand confidently with Catherine’s message about contributing towards a greener future and making thoughtful decisions to this end, and we are excited that this was very well-received by the audience.
On second day, our founder and CEO, Cédric Villemain, presented on the statistics-based highlights of the PostgreSQL major release version 18 (official notes linked here). (After the event, he created a blog post on the topic of Cumulative Statistics in Postgres 18 - check it out here!) Our team here at Data Bene are a fan of stats and surfacing data, which is what led us to create the PostgreSQL extension, StatsMgr. If you previously missed information on StatsMgr and would like to learn more, check out our previous blog and Postgres Café podcast episode on the topic.
Directly after Cedric, I had the opportunity to present on developing and maintaining a Rust-based PostgreSQL extension (currently still very early stage) that helps facilitate the transition from Oracle to PostgreSQL. This project, named pgtap_gen, leverages the pre-existing pgTAP project to automate unit test creation on IvorySQL (100% open-source forked PostgreSQL with Oracle-analogous components). It was very encouraging to hear audience engagement and feedback. What’s more, it feels great to participate in the PostgreSQL ecosystem, even if it’s for proof-of-concept initiatives!
Towards the end of the second day, Frédéric delievered a talk on transaction isolation levels in PostgreSQL (that you may get the chance to see later on, as he’s submitted the same talk to pgDay Paris 2026 and PGDay CERN 2026!). He delved into each isolation level in Postgres with practical examples, and explained how everything works behind the scenes to define real steps for confidently choosing the right level for the right context. The attendees seemed to find the talk interesting and educational, so we may yet see further content on this topic and similar to help others who missed the opportunity to learn about the subject.
We chose to sponsor and attend PGIBZ 2025 as we believe it is important to further the PostgreSQL community across the world. No matter what, this is an event that is seeking to further Postgres education for the better. Profits from the event are used by the parent nonprofit organization, PostgreSQL España, to invest in translations of the official PostgreSQL documentation into Spanish.
Further breakdowns of each of the talk topics mentioned above are in-the-works; feel free to follow us on LinkedIn and Mastodon to keep an eye out for them when they’re published on our blog.
We very much enjoyed our time at PGIBZ 2025, and are honored to have had the opportunity to present and participate. Looking forward to returning next year!
More and more companies are seeking to free themselves from the costs and constraints of Oracle by migrating to PostgreSQL. But how can this transition be achieved without spending months on it? Here is a proven, fast and secure method, based on feedback and open source tools.
Before migrating, it is essential to take stock of the situation: we start by listing Oracle objects through performing a complete extraction (including tables, views, sequences, procedures, functions, and packages).
It is also important to communicate regularly with the client to refine findings and validate directions, as well as to identify dependencies (applications connected with which technology, jobs, interfaces, etc.).
Finally, identify the client’s Oracle-specific features (specific functions such as ROWID, DUAL, DBMS_XXX, etc.) and any other keywords requiring special attention.
The success of a transition to PostgreSQL – because it is indeed a gradual transition and not a migration – depends as much on strategy as it does on tools.
It is not enough to simply copy objects: you need to understand the subtleties of the Oracle engine and intelligently transpose them into the PostgreSQL universe.
Among the essential tools, Ora2Pg plays a central role. Not only does it generate clear inventory reports, it also automates the conversion of schemas, data and PL/SQL code. It is a valuable accelerator for well-structured projects.
But in some contexts, the reality is more complex. Many customers accumulate significant technical debt, with thousands of lines of procedural code. Attempting to rewrite everything in PL/pgSQL in a short period of time would exhaust teams and compromise the migration itself. Maintaining two code bases— one in PL/SQL, the other in PL/pgSQL— during a complete transition phase is often organisationally untenable.
This is precisely where IvorySQL offers a powerful alternative. This fork of PostgreSQL, enhanced with Oracle compatibilities (PL/SQL, ROWID, DUAL, etc.), significantly reduces the need for immediate rewriting. By combining IvorySQL with targeted documentation and business validation scripts, the transition is easier, more reliable, and above all, more controlled.
Once in production, teams have the time they need to gradually rewrite their procedural and remaining application code in PL/pgSQL, without pressure or functional disruption.
One of the great advantages of IvorySQL is that it performs part of the integration upstream. Thanks to its native support for PL/SQL, Oracle objects and Oracle-compatible syntax, many elements migrated via Ora2Pg work immediately without any major adaptation.
Of course, this does not eliminate the need for testing. On the contrary, it is essential to validate business behaviour in an IvorySQL pre-production environment. Procedures, functions, packages, triggers, and views must be tested with real-world cases, comparing the results between Oracle and IvorySQL to ensure functional equivalence.
Comparison scripts can be used to verify query responses, transaction effects, or the behaviour of migrated functions. But in many cases, IvorySQL significantly reduces the need for patches or rewrites, speeding up production and securing the transition.
A successful transition is not limited to technical aspects; it also relies on clear documentation and appropriate support. It is essential to write comprehensive test specifications for each complex object and module (procedure, package, trigger, etc.) in order to keep track of technical choices, adaptations made, and points of attention.
Support for client teams is just as important. Targeted workshops are used to share best practices, explain the differences between Oracle and PostgreSQL, and facilitate the adoption of IvorySQL.
Finally, a successful transition to PostgreSQL also depends on support for technical and functional teams. Targeted training sessions enable developers, DBAs, and project managers to learn the specifics of IvorySQL, understand the differences with Oracle, and quickly develop their skills.
This system also relies on structured technical support, which helps resolve issues, secure implementation choices, and facilitate communication between project teams and migration experts.
During a recent assignment for a private sector company, we supported the transition from an Oracle database to IvorySQL in a challenging context: where there was significant technical debt, high volume, and tight deadlines.
Thanks to IvorySQL and our ability to automate this transition, it was completed quickly and almost all objects were integrated without rewriting.
The switch from IvorySQL to PostgreSQL can be carried out smoothly, as long as the application is fully functional and meets business expectations.
The teams were able to focus on business testing, while targeted training and technical support enabled rapid skill development.
It is essential to succeed in this type of project in order to build confidence and roll out this operation across your entire Oracle instance fleet!
Are you considering a transition from Oracle to PostgreSQL? Let’s talk about it.
Contact our team of experts at any time: contact@data-bene.io
On November 26, 2025 IvorySQL 5.0 was released - view the announcement here for more information about this open-source project.
]]>The PostgreSQL documentation Appendix K states a table can have a maximum of 1,600 columns.
This is a hard coded limit that can be found in the source code at src/include/access/htup_details.h:
#define MaxTupleAttributeNumber 1664
#define MaxHeapAttributeNumber 1600Let’s fully validate the claim and test accordingly.
Here, we’ll use a simple bash script because it is easy to adapt while testing.
-- Classic example
DO $$
DECLARE
i int;
BEGIN
EXECUTE 'DROP TABLE IF EXISTS tint_1601;';
EXECUTE 'CREATE TABLE tint_1601(i_1 int);';
FOR i IN 2..1601 LOOP
EXECUTE format('ALTER TABLE tint_1601 ADD COLUMN i_%s int;', i);
END LOOP;
END $$;The typical output is as follows:
NOTICE: table "tint_1600" does not exist, skipping
ERROR: tables can have at most 1600 columns
CONTEXT: SQL statement "ALTER TABLE tint_1600 ADD COLUMN i_1601 int;"
PL/pgSQL function inline_code_block line 8 at EXECUTESo far so good (or at least, all is working as expected).
You might have the idea to try replacing int4 with int2 type to create a 1,600+ column table. It will not work as this is a hard coded limit.
Let’s build a 1,600 column table with the same demonstrated code.
DO $$
DECLARE
i int;
BEGIN
EXECUTE 'DROP TABLE IF EXISTS tint_1600;';
EXECUTE 'CREATE TABLE tint_1600(i_1 int);';
FOR i IN 2..1600 LOOP
EXECUTE format('ALTER TABLE tint_1600 ADD COLUMN i_%s int;', i);
END LOOP;
END $$;Another sql script can be used to produce a valid 1,600 column tuple:
DO $$
DECLARE
s TEXT;
rows_inserted int;
BEGIN
s := format(
'INSERT INTO tint_1600 VALUES (1%s);'
, repeat( ',1' , 1599 )
);
EXECUTE s;
GET DIAGNOSTICS rows_inserted = ROW_COUNT;
RAISE NOTICE 'Rows inserted: %', rows_inserted;
END $$;The output is:
NOTICE: Rows inserted: 1
DOAnother success with no surprise.
Let us continue pushing to the limits.
We now create another 1,600 column table using the char(127) data type.
We reuse our sql script with some modifications:
-- Create a table with 1,600 columns: 1 x int + 1599 x char(127)
DO $$
DECLARE
i int;
BEGIN
EXECUTE 'DROP TABLE IF EXISTS tint_1600;';
EXECUTE 'CREATE TABLE tint_1600(i_1 int);';
FOR i IN 2..1600 LOOP
EXECUTE format('ALTER TABLE tint_1600 ADD COLUMN c_%s char(127) NOT NULL;', i);
END LOOP;
END $$;
-- Insert a tuple - 1 x int + 1599 x char(127)
DO $$
DECLARE
s TEXT;
BEGIN
s := format(
'INSERT INTO tint_1600 VALUES (1%s);'
, repeat( $q$,'1'::char(127)$q$ , 1599 )
);
EXECUTE s;
END $$;The output is:
ERROR: row is too big: size 25616, maximum size 8160As we can see, the table has 1,600 columns but this time the tuple cannot fit a single heap page which explains the error “row is too big: size 25616, maximum size 8160”. If you paid attention to the modified script, you can see columns are defined as NOT NULL so at table creation PostgreSQL could have proven data insertion was impossible.
To keep things simple, let us auto-join:
SELECT a.*,b.* FROM tint_1600 a, tint_1600 b;
ERROR: target lists can have at most 1664 entriesNow the SELECT clause (a.*,b.*) is reaching its own limit (MaxTupleAttributeNumber = 1664).
Sometimes, you have to modify your application and it generates schema modifications.
Most of the time, there are table modifications like adding or dropping columns.
ADD / DROP COLUMNLet us see what happens from the SQL side when we add, then drop, a column.
=# CREATE TABLE tadc_1600(i_1 int NOT NULL);
CREATE TABLE
=# ALTER TABLE tadc_1600 ADD COLUMN i_2 int NOT NULL;
ALTER TABLE
=# SELECT attname,attnum,attstorage,attnotnull,attisdropped
FROM pg_attribute
WHERE attrelid=(
SELECT oid
FROM pg_class
WHERE relname='tadc_1600'
)
AND attnum > 0 ORDER BY attnum;
attname | attnum | attstorage | attnotnull | attisdropped
---------+--------+------------+------------+--------------
i_1 | 1 | p | t | f
i_2 | 2 | p | t | f
(2 rows)
=# ALTER TABLE tadc_1600 DROP COLUMN i_2;
ALTER TABLE
=# SELECT attname,attnum,attstorage,attnotnull,attisdropped
FROM pg_attribute
WHERE attrelid=(
SELECT oid
FROM pg_class
WHERE relname='tadc_1600'
)
AND attnum > 0 ORDER BY attnum;
attname | attnum | attstorage | attnotnull | attisdropped
------------------------------+--------+------------+------------+--------------
i_1 | 1 | p | t | f
........pg.dropped.2........ | 2 | p | f | t
(2 rows)When dropping a column,
One can wonder if there is a limit to the number of add/drop operations that can be run on a given table.
As usual, let us try:
-- ADD / DROP COLUMN example
DO $$
DECLARE
i int;
BEGIN
EXECUTE 'DROP TABLE IF EXISTS tadc;';
EXECUTE 'CREATE TABLE tadc(i_1 int);';
FOR i IN 2..1601 LOOP
EXECUTE format('ALTER TABLE tadc ADD COLUMN i_%s int;', i);
EXECUTE format('ALTER TABLE tadc DROP COLUMN i_%s;', i);
END LOOP;
END $$;The output is:
ERROR: tables can have at most 1600 columns
CONTEXT: SQL statement "ALTER TABLE tadc ADD COLUMN i_1601 int;"
PL/pgSQL function inline_code_block line 8 at EXECUTEOh oh! We reached the 1,600 limit here as well.
Let us explore a bit after add/drop column 1,599 times:
=# SELECT attname,attnum,attstorage,attnotnull,attisdropped
FROM pg_attribute
WHERE attrelid=(
SELECT oid
FROM pg_class
WHERE relname='tadc'
)
AND attnum > 0 ORDER BY attnum;
attname | attnum | attstorage | attnotnull | attisdropped
---------------------------------+--------+------------+------------+--------------
i_1 | 1 | p | t | f
........pg.dropped.2........ | 2 | p | f | t
........pg.dropped.3........ | 3 | p | f | t
........pg.dropped.4........ | 4 | p | f | t
........pg.dropped.5........ | 5 | p | f | t
........pg.dropped.1599........ | 1599 | p | f | t
........pg.dropped.1600........ | 1600 | p | f | t
(1600 rows)Well, table tadc has 1,600 columns. You can see this as modifications are appending and table content rewriting is avoided.
At this point, further column add & drop modifications will fail.
Is there anything I can do to escape this situation?
The VACUUM command operates at the tuple level so even if you run a VACUUM FULL the table structure will not change.
This is not an issue with dead tuples but rather an issue with the catalog.
You’ll need to create a new table definition.
Here are some solutions, from simple to complex:
Build a new table (requires service downtime)
CREATE TABLE (LIKE INCLUDING ALL)COPY data from old to new tableLeverage logical replication (minimize service downtime)
CREATE TABLE LIKE (INCLUDING ALL)CREATE local PUBLICATION/SUBSCRIPTIONThe above solution works fine for simple cases. But real life tables often
use integrity constraints. Let’s explore a bit using foreign keys.
-- Foreign key case
=# CREATE TABLE colors (id int, name text );
=# CREATE TABLE objects ( id int, color_id int, name text );
=# ALTER TABLE colors ADD PRIMARY KEY (id);
=# ALTER TABLE objects ADD CONSTRAINT fk_color
FOREIGN KEY (color_id) REFERENCES colors (id);
=# INSERT INTO colors
VALUES (1,'red'), (2, 'green'), (3, 'blue' );
=# INSERT INTO objects
VALUES (1,1, 'red object')
,(2,2, 'green object')
,(3,3,'blue object');Let’s apply the recipe:
-- Duplicate table structure (valid columns only) and copy data
=# CREATE TABLE tmp_colors (LIKE colors INCLUDING ALL);
=# INSERT INTO tmp_colors SELECT * FROM colors;
-- Do the DROP/RENAME trick
=# BEGIN;
=# DROP TABLE colors;
=# ALTER TABLE tmp_colors RENAME TO colors;
=# COMMIT;The DROP TABLE command issued an error:
ERROR: cannot drop table colors because other objects depend on it
DETAIL: constraint fk_color on table objects depends on table colors
HINT: Use DROP ... CASCADE to drop the dependent objects too.As we can see, the recipe has to be changed to include dependent tables as well.
Adding CASCADE will drop FK constraints on dependent tables.
Let’s run a modified version of the recipe:
-- Do the DROP/RENAME trick
=# BEGIN;
=# DROP TABLE colors CASCADE; -- DROP related FOREIGN KEY constaints
=# ALTER TABLE tmp_colors RENAME TO colors;
-- Recreate FK contraint
=# ALTER TABLE objects ADD CONSTRAINT fk_color
FOREIGN KEY (color_id) REFERENCES colors (id);
COMMIT;We have to check the behaviour is the expected one:
=# INSERT INTO objects VALUES (5,5,'ro');
ERROR: insert or update on table "objects" violates foreign key constraint "fk_color"
DETAIL: Key (color_id)=(5) is not present in table "colors".
=# INSERT INTO objects VALUES (5,3,'ro');
INSERT 0 1Success!
When integrity constraints are too numerous or you find it difficult to follow,
you may use pg_dump/pg_restore to rebuild all automatically. If service downtime
is an issue, you may use logical replication to perform like pg_dump/pg_restore.
As you can see, having to deal with the 1,600 column limit is not something you would
like to do just for fun (usually). Notably, it can lead to service downtime.
Do you have other ideas of how to address this situation? Have you run into odd ways of reaching this hard-coded limit? Contact us! We always love a good discussion about PostgreSQL.
]]>Statistics (in the broad sense: monitoring counters, I/O metrics, and planner / optimizer estimates) lie at the heart of both performance tuning and internal decision making in PostgreSQL. Transparent, reliable, and manipulable statistics, among other things, allow DBAs to address the efficiency of PostgreSQL directly, as well as enable “extensions” to improve the user experience.
That said, the historic statistics system of PostgreSQL has not been without points of friction. These include limited ability to clear (relations) statistics, metrics with units that don’t always align with user goals, and no C API for using the PostgreSQL Cumulative Stats engine. PostgreSQL 18 addresses these concerns head on.
Below is a summary of the key enhancements.
While statistics offer incredible value, their collection can take up significant time and resources. PostgreSQL 18 introduces an important consideration: with the expanded range of collectible metrics, the hash table maximum size has been increased. Do keep in mind, especially if you’re designing large-scale systems with table-per-customer architectures, that 1GB ceilings have been shown to be hit with some millions of tables.
Here are the major new or improved features relating to statistics and monitoring. Each item links to the relevant documentation or code where possible.
Generally, pg_stat_io now reports I/O activity in bytes rather than pages, which is more convenient for analysis. Moreover, WAL statistics were moved here from pg_stat_wal, providing a single, comprehensive view.
pg_upgrade is now able to retain optimizer statistics, removing the need to run a full ANALYZE on the databases to get good planning of queries after the upgrade; this is a very welcome update for large databases! Be aware that custom statistics added by an extension along with those created with CREATE STATISTICS won’t be retained.
You will surely want to look at new options in vacuumdb (--missing-stats-only) to, well, analyze only what’s needed.
On a similar note, the --[no-]statistics flag has been added to pg_dump, pg_dumpall, and pg_restore.
It’s now easier to know the maintenance effort on objects with total time spent on VACUUM and ANALYZE operation (and automatic ones) now reported into pg_stat_all_tables and variants.
A new GUC to not forget is track_cost_delay_timing. It collects time spent sleeping (due to delayed operations) for VACUUM and ANALYZE. While very interesting, like other track_io* GUCs, it implies a lot of extra calls to the system clock which on some platforms can lead to a severe performance impact. Always check with tool like pg_test_timing to ensure your system can afford it!
No more questions about checkpointer activity when using pg_stat_checkpointer. The new attribute num_done lets us know the number of completed checkpoints. You can also get what kind of buffers were written with slru_written and buffers_written now only matching shared_buffers: previously log and view were not providing the same counts because there was a SLRU counter in one case and not the other.
Want to know more about the I/O handled by the backend (PID)? Call pg_stat_get_backend_io(int) and you’ll get output similar to what the pg_stat_io view provides, for this process (excluding those already). As for the WAL stats for this PID: call pg_stat_get_backend_wal(int).
New attributes parallel_workers_to_launch and parallel_workers_launched were introduced in pg_stat_database. The ratio lets us know if we have enough slots for parallel workers.
Interesting changes on pg_stat_statements: more queries will be grouped under the same identifier. For example, patterns IN (1,2,3, ...) as only first and last constant will be used. A more counter-intuitive change is related to the table name used in a query. Only the name is used, not the schema or relation OID. This last change allows us to track dropped or recreated tables for example, but it will group statistics from unrelated tables if they have just the same name. The way to keep separate statistics for tables with same name is to alias them in the queries (FROM my.table mt, other.table ot)…
Finally, additions to pg_backend_memory_contexts with path (to get parent/child) and type to segregate AllocSet, Generation, Slab and Bump contexts… and what exactly are Slab and Bump? They are not documented; for these you’ll want to read headers of C files here. They exist to optimize memory allocation, reallocation, and reset, depending on expected memory usage. For example, Slab is defined as a «MemoryContext implementation designed for cases where large numbers of equally-sized objects can be allocated and freed efficiently with minimal memory wastage and fragmentation».
Ah, no, a last one, wal_buffers_full was added to pg_stat_statements to allow us to tune for wal_buffers with better insights.
There are now better insights for conflict management when using logical replication that leverage new attributes in pg_stat_subscription_stats. As reference, this excerpt from the commit entry lists the following attributes that were introduced:
confl_insert_exists:
Number of times a row insertion violated a NOT DEFERRABLE unique
constraint.
confl_update_origin_differs:
Number of times an update was performed on a row that was
previously modified by another origin.
confl_update_exists:
Number of times that the updated value of a row violates a
NOT DEFERRABLE unique constraint.
confl_update_missing:
Number of times that the tuple to be updated is missing.
confl_delete_origin_differs:
Number of times a delete was performed on a row that was
previously modified by another origin.
confl_delete_missing:
Number of times that the tuple to be deleted is missing.
There is now a new set of functions to manage relation and attributes stats (relpages, avg_width, and so on). This gives you the freedom to export, import, and adjust stats as you want, so you can replicate planner behavior outside of “production”, maintain patched stats, and so on.
One of the most exciting parts is what PostgreSQL 18 opens up for extension authors.
This tiny line at bottom of section E.1.3.9 Modules is what concerns these changes:
Allow extensions to use the server’s cumulative statistics API (Michael Paquier)
Previously statistics manipulation was an internal-only affair; now there is an official, structured API surface you can build on (or wrap).
The commit message is well written, and covers most of the new functionality. A subset of the options is detailed in the documentation. However, you will need to go into source code to know more at this stage; in particular, it’s worth having a look at the injection points extension (provided in core) which uses the new API.
For a deeper dive into how an extension can leverage these new capabilities, soon you will be able to see PACS (PostgreSQL Advanced Cumulative Statistics) on Codeberg - my project that provides a wrapper library and helper utilities around the new PostgreSQL 18 statistics APIs.
In the meantime, the talk I gave at FOSDEM 2025 explores these topics in greater detail.
]]>The survey results show that PostgreSQL is ranked the highest among all database technologies for developers that want to use it in the next year (47%) or have used it this year and want to continue using it next year (66%) for the third year in a row.
The survey data from over 49,000 developers across 177 countries provides clear evidence of PostgreSQL’s sustained appeal. Since 2023, PostgreSQL has consistently ranked as both the most desired and most admired database technology among developers.
Looking at the specific metrics from the survey visualizations, PostgreSQL leads with 46.5% of developers wanting to work with it in the coming year, while an impressive 65.5% of those who have used it want to continue doing so. These aren’t just impressive numbers—they represent a consistency that’s rare in the rapidly changing technology landscape.
The survey data also reveals an interesting pattern among developers currently using other database technologies. Developers working with MongoDB and Redis show a particularly strong desire to add PostgreSQL to their toolkit next year, seeing the value in adding relational database skills to their repertoire.
Why has PostgreSQL achieved this level of sustained success? The answer lies in its community-driven development model. As an open source project, PostgreSQL benefits from collaborative development that is both transparent and responsive to real-world needs.
The PostgreSQL project represents the best of what community-driven development can achieve. With over 400 code contributors across more than 140 supporting companies, the project boasts over 55,000 commits and more than 1.6 million lines of carefully crafted code. This diverse, globally distributed approach to development results in more thorough testing, faster bug fixes, and more innovative features than traditional commercial development models typically achieve.
Major versions are released annually with approximately 180 features per release, complemented by quarterly minor releases that include numerous improvements and fixes. This steady cadence of innovation consistently contributed by individuals all over the world ensures PostgreSQL doesn’t just keep pace with developer needs—it anticipates them. More than that, every individual has the agency to contribute to the project to ensure that anywhere the software is lagging behind, functionality changes to address modern demands.
One key factor in PostgreSQL’s broad appeal is that it’s not limited to being just a relational database system. PostgreSQL is object-relational by design, capable of handling diverse data types including JSON/JSONB, XML, Key-Value, geometric, geospatial, native UUID, and time-series data. This versatility explains why developers from NoSQL backgrounds find PostgreSQL attractive—it offers relational reliability while maintaining the flexibility they’re accustomed to.
The extensive support for different data types, combined with ACID (Atomicity, Consistency, Isolation, Durability) characteristics, enables optimized, performant, and reliable data handling regardless of the specific requirements in place. Additionally, PostgreSQL’s huge community-driven extension network builds on its native extensibility, providing solutions for geospatial handling, disaster recovery, high availability infrastructure, monitoring, auditing, and much more.
While PostgreSQL dominates the top positions, the survey reveals a healthy, competitive database ecosystem. The complete rankings show:
Most Desired Databases:
Most Admired Databases:
These numbers reflect a diverse ecosystem where different databases serve specific purposes. SQLite’s strong performance highlights the continued importance of lightweight, embedded solutions. Redis maintains its position as a highly regarded specialized database for caching and real-time applications. Traditional databases like MySQL and Microsoft SQL Server continue to hold significant positions, while newer technologies like DuckDB show impressive admiration scores despite lower usage rates.
Three consecutive years at the top of developer preferences doesn’t happen by accident. PostgreSQL’s sustained dominance stems from fundamental strengths that continue to serve developers well as technology landscapes shift. The resilience built into PostgreSQL through its community-driven development model means it adapts without losing stability. Its extensibility sets it apart in practical ways—rather than waiting for vendor roadmaps or worrying about feature gaps, developers can build what they need or leverage the extensive ecosystem of community extensions. The open source nature ensures PostgreSQL remains focused on developer needs rather than business models, with bug fixes happening quickly and features developing based on real-world use cases.
After 35 years of active development and three consecutive years as the most desired database technology, PostgreSQL has proven that community-driven open source development can deliver both immediate utility and long-term value. For developers and organizations looking at their database choices, PostgreSQL offers something increasingly rare: a technology that gets better over time without leaving its users behind.
]]>The program featured 101 technical talks led by 105 speakers. The majority of sessions were in Mandarin, with an English track offered with simultaneous translation. You can view the full program at IvorySQL.io and find the replays on Weibo via this link.
There was a small group of the international community present, including Grant Zhou, who liaises and collaborates with the PostgreSQL association in China and the rest of the world.
On a technical level, the content was dense and particularly interesting. I particularly noted:
Alena Rybakina’s presentation on the PostgreSQL query planner and strategies for circumventing certain limitations.
A clear and concrete focus on Patroni (High Availability) by Alexander Kukushkin and Polina Bungina.
Florentz Tselai presented two topics applying his principles: simplicity and efficiency, using “AI” with PostgreSQL, and data management with Sun Tzu and the 36 Stratagems as support.
Also a very good introduction to Bazel and its use for Monogres (to be officially announced soon) presented energetically by Alvaro Hernandez.
Monogres is a very interesting initiative that should help strengthen control over the software supply chain, a major theme in IT today. And I also saw it as a great opportunity to showcase PostgreSQL variations with features and fixes that aren’t always possible to include in PostgreSQL itself or backport to previous major releases.
Michael Meskes had the honor of giving a plenary lecture on a topic that richly deserves it: “From Code to Commerce: Open Source Business Models Today,” a keynote on open-source and free software, applied to the PostgreSQL ecosystem.
My colleague Andrea presented the developments and trends of companies moving to IvorySQL and PostgreSQL.
For my part, I presented Linux PSI in the PostgreSQL context.
Since everything is recorded, I encourage you to explore and watch the topics that interest you. There were also pre-recorded lectures in English during the event, but I admit that I took advantage of the time during the sessions to interact with participants.
Aside from the conferences, I had the chance to meet several members of the Chinese PostgreSQL community who are very well-known for their involvement in the success of PostgreSQL locally. I also had the opportunity to learn more about Cloudberry, replacing Greenplum, thanks Dianjin Wang!
Data Bene also planned a time to meet with the IvorySQL team, based largely in Shandong, the province where Jinan, the host city of the conferences, is located. Ivory is a project in which we are actively involved and which allows companies to move away from Oracle “smoothly.” This is an important topic for our clients and one that occupies a prominent place in our partnership with Highgo: they have been working on this project for several years now, and we want to enable companies everywhere to benefit from it with appropriate support and expertise.
The conferences were very well organized and the welcome was wonderful, the “Social Event” at the local “beer garden” perfectly suited to the heat of Jinan at the end of June!
Given the conference program, I bitterly regretted not understanding anything (there was 1 track in English and 5 in Mandarin)… but it is already being said that next year the Mandarin conferences could perhaps be translated (into English), and the date brought forward to May to take advantage of a milder climate.
There is so much to learn there that I will gladly return.
]]>At the conference I had the opportunity to present on Postgres within the context of the life sciences (discussed in the next section). And, altogether, I felt this conference had a nice diversity of talks: a selection that spanned Postgres core, its ecosystem, and beyond.
I’m confident that by the end, most if not all attendees left more enriched in some way relative to when they arrived.
Leading up to this event I had the honor of one of my talks being accepted. The title was “Postgres and Life Science: From Cells to Stars” and it was organized as a meta-analysis / homage to the extensibility of Postgres and its various applications to the natural world.
In order to best tell this story, I walked the audience through the following five topics of increasing scope:
I enjoyed putting together and presenting the talk, and there was nice discussion afterwards. Two points stood out in particular that I felt would be interesting to address here:
What three technologies (tools / workflows) would benefit most greatly, in terms of increased impact or adoptability, if their complexities were significantly reduced / abstracted away?
During my talk I made a claim that the brain was ACID compliant. While I was referring mostly to the action potentials of neurons, this was rightfully challenged.
1. What three technologies (tools / workflows) would benefit most greatly, in terms of increased impact or adoptability, if their complexities were significantly reduced / abstracted away?
Right out of the gate I thought about magnetic resonance scanner image classification. There’s quite a lot of conversation surrounding this topic within the medical community and there are plenty of startups in this space as well. My personal opinion is that there is momentum in the direction of accessibility, but there is still a strong separation between developer and end user. While I don’t know the answer at this point, I would look into pgvector and postgresml as a starting point. Due to this challenge’s involvement of vectors and machine learning, I would consider leveraging an image embedding service to turn the raw MRI output into a format that pgvector might be able to work with.
As a former academic, I can speak to the ubiquitousness of the common spreadsheet (.csv format being less common, but still utilized). What’s more is that files are typically stored in local directories / private server / shared infrastructure, but nevertheless a vanilla folder architecture. One can imagine the potential frictions as the conversation scales to include multiple researchers across multiple groups. Factor in a naturally high student turnover, paired with an “I like doing it my way” mentality, and one could appreciate the value of standards. While improvements could be approached from a number of different angles, I’d like to focus on data management and version control.
Tidy data and good organizational hygiene are hallmarks of success in any field of study. However, tracking changes are most often, if not exclusively limited to text documents. While it might be surprising to the reader, “code repository” is not part of the common academic lexicon. Even the term “Linux” evokes an air of “mysterium tremendum et fascinans” (Otto, 1923). With data security at the top of the mind, self hosted options such as forgejo could potentially benefit life scientists greatly - particularly if there are reservations about storing data online. Instead of having multiple file drafts, e.g., “draft-1_final”, “draft_final_final”, etc, tools such as forgejo can help track progress and give researchers more transparency into past changes (leading into easier cross-team collaboration).
Trust is a central topic in any field of research, and in certain circumstances, auditing (or otherwise some form of proof of work) may take center stage. In this case, Postgres and one of its companion extensions, pgaudit, can offer a nice step towards compliance. Due to Postgres’ capabilities, it can sometimes be viewed as intimidating and only suitable for large projects. I think there could be a lot of ubiquity with the publication of a “Postgres for small scale projects” type guide.
At the end of the day, no one will willingly use something unless they know it exists. That’s why discoverability is one of the most fundamental concepts when discussing impact and adoptability. It’s up to the maintainers, contributors, and communities behind these open source tools to share what they’re up to on multiple platforms, as well as different conferences. Honestly, the easiest way to help is to just talk about it and get hands-on.
2. During my talk I made a claim that the brain was ACID compliant. While I was referring mostly to the action potentials of neurons, this was rightfully challenged.
This was another exciting conversation in the post-presentation discussion, and while this really warrants its own blog post, I wanted to quickly share my thoughts. Within one of my slides, I made the claim that the brain is ACID compliant, at least in the sense of transactions being all-or-nothing. Neurons, which are a common cell type in the brain, have a characteristic whereby they receive signals which compile until a threshold is reached and then the neuron sends a signal of its own, or “fires.” This is a gross oversimplification: here’s a quick Wikipedia link for more information.
However, astute audience members righty noted that the brain is complex and has different regions. There is memory loss and there are activities that can alter function and consciousness. However, to what extent do external influences on the brain correspond to a database system? If something corrupts a Postgres database, it is no longer ACID compliant, but it was beforehand. All these points are both valid and interesting. It will be interesting to think on this and write a more formal response.
To sum things up, this was great conference. I know I speak for all attendees when I extend a thank you to all involved, whether they be staff, volunteers, speakers, or otherwise.
Foote, K. J., Grant, J. W. A., & Biron, P. M. (2024). A global dataset of salmonid biomass in streams. Scientific data, 11(1), 1172. https://doi.org/10.1038/s41597-024-04026-0
Giordano, C., & Hadjibagheri, P. (2021, December 11). UK COVID-19 dashboard built using Postgres and Citus for millions of users. Microsoft TechCommunity Blog. https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/uk-covid-19-dashboard-built-using-postgres-and-citus-for/ba-p/3039052
Kazimiers, T., et al. (2021). CATMAID (Collaborative Annotation Toolkit for Massive Amounts of Image Data) [Computer software]. GitHub. https://github.com/catmaid/CATMAID
Krefl, D., & Nienartowicz, K. (2025, January 17). Harnessing Postgres and HPC for petabyte-scale variable star classification in astronomy [Conference presentation]. CERN PGDay 2025, Geneva, Switzerland. https://indico.cern.ch/event/1336647/contributions/5660229/
Otto, R. (1923). The idea of the holy: An inquiry into the non-rational factor in the idea of the divine and its relation to the rational (J. W. Harvey, Trans.). Oxford University Press. (Original work published 1917)
Teixeira, A. de A., & PgHydro Project. (2022). pghydro (Version 6.6) [Computer software]. GitHub. https://github.com/pghydro/pghydro
Wikipedia contributors. (2025, May 16). Action potential. Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/wiki/Action_potential
]]>This year’s conference focused around the topics of AI, DevOps and cloud-native technologies, open source community engagement, security and compliance, systems and infrastructure, and FOSS @ home (exploring the world of self-hosted applications and cloud services).
The conference drew attendees from around the world to talk about everything open-source, revolving around Linux at the core (of course) while continuing the discussion across topics such as embedded systems & IoT. As always, there was a unique blend of cutting-edge tech talk and practical problem-solving within every space that is what makes SCaLE special.
PostgreSQL@SCaLE22x ran as a dedicated two-day, two-track event on March 6-7, 2025, recognized under the PostgreSQL Global Development Group community event guidelines. The selection team included Gabrielle Roth, Joe Conway, and Mark Wong, ensuring the quality you’d expect from the PostgreSQL community.
The speaker lineup was impressive: Magnus Hagander, Christophe Pettus, Peter Farkas, Devrim Gündüz, Hamid Akhtar, Henrietta Dombrovskaya, Shaun Thomas, Gülçin Yıldırım Jelínek & Andrew Farries, Nick Meyer, and Jimmy Angelakos. One particularly memorable session was titled “Row-Level Security Sucks. Can We Make It Usable?”—a refreshingly honest take on PostgreSQL’s RLS feature that probably resonated with more than a few database administrators in the audience.
The community “Ask Me Anything” panel was hosted by Stacey Haysler and featured Christophe Pettus, Devrim Gündüz, Jimmy Angelakos, Magnus Hagander, and Mark Wong. These sessions are where the real knowledge transfer happens—no marketing speak, just practitioners talking shop about PostgreSQL internals, performance, best practices, and the future of the database.
Behind the scenes, volunteers Derya Gumustel, Erika Miller, Hamid Akhtar, Jennifer Scheuerell, Mark Wong, and Roberto Mello kept everything running smoothly, with PGUS hosting the booth in the expo hall.
Personally, I had the pleasure of collaborating with Jimmy Angelakos during his live streaming sessions featuring other guests like Henrietta Dombrovskaya, Mark Wong, Gülçin Yıldırım Jelínek, and even a brief cameo from Devrim Gündüz.
One of the topics discussed with Gülçin Yıldırım Jelínek on the podcast is whether or not there’s any community interest in continuing Postgres Café. What do you think? Do you want to see more episodes from this podcast series, expanding discussions on extension and open source development to the rest of the community and beyond? Let us know: contact@data-bene.io
There are a lot of co-located events besides PostgreSQL @ SCaLE, including “SCaLE: The Next Generation (TNG)” which is a youth-focused tech event encouraging interactive activities and presentations for students, and the annual Cybersecurity Capture the Flag (CTF) game event presented by Cal Poly FAST and Pacific Hackers.
SCaLE remains an excellent place to network when looking to advance your career in open source. Socializing at the booths is always an excellent way to make connections and find opportunities, of course, but Open Source Career Day also returned in order to offer a dedicated space for professionals and aspiring technologists to become empowered with resources, tools, real-world examples, and engaging content from presentations and workshops.
The fun tradition of holding a Saturday Game Night with food & drinks also continued this year, with Trivia Night (presented by Uncoded) and other fun activities such as inflatable axe throwing, nerf target practice, arts & crafts, a board game room, casino night, & a blocks room for building derby cars, playing pictionary, or building with large blocks.
SCaLE has established itself as a consistent presence in Pasadena, and this stability has allowed the conference to build lasting relationships with the local community and venues. Keep an eye out for SCaLE 23x announcements - it promises to be well worth the visit.
For those interested in PostgreSQL@SCaLE specifically, stay tuned to the PostgreSQL mailing lists for announcements about volunteering, speaking opportunities, or other ways to participate in next year’s event. The PostgreSQL track and booth is a consistent source of engaging discussions amongst those in the Postgres community and beyond, reflecting the database’s growing adoption across industries.
In a world where many tech conferences feel more like elaborate vendor showcases, SCaLE remains that rare gathering where community comes first, collaboration is genuine, and the technology discussions are driven by practitioners solving real problems. Mark your calendars for SCaLE 23x—this is one conference that consistently delivers on its promise of bringing together open source enthusiasts to actually collaborate and learn.
Wish you hadn’t missed out? You can always check out the YouTube playlist of talks that were recorded during the conference to at least benefit from the knowledge contained therein.
]]>Once upon a time, in a faraway server room encased in heavy glass and reinforced concrete, lived a PostgreSQL database so confined, so secluded, it could only dream of the cloud.
No network.
No USB.
No writable external storage device.
Just a keyboard, a monitor, and the hum of industrial-grade air filters.
This wasn’t your average air-gapped setup. This was a zero-exfiltration zone, with operational security so tight you’d think it was guarding state secrets—or worse, legacy banking software.
And yet, in this digital oubliette, one innocent challenge remained:
How do you back up a PostgreSQL database without ever extracting a file?
Our customer didn’t just want backups—they needed them. The fear wasn’t theft, it was total failure: a motherboard dying quietly in its glass sarcophagus, taking the data with it. And if it came to that, chiseling through reinforced architecture wasn’t a viable disaster recovery plan.
We brainstormed everything:
psql output? Way too verbose.And then came the epiphany: QR codes.
What if we could pg_dump the database…
into QR codes…
on the screen…
captured by a high-speed camera…
then reassembled frame by frame outside the vault?
It was so absurd, it just might work.
pg_dump: Now with More PixelsPostgreSQL’s beloved pg_dump tool is modular. So we extended it with a custom archiver: --format=qrcode.
Here’s how it works:
QR encoding: Each chunk of SQL output is encoded into a PNG QR code.
Streaming: Instead of saving to disk, we push the stream of PNGs directly to stdout.
Framing: Our UI lays out multiple QR codes on a single screen using high-DPI output. (We’re talking 2000+ pixels here—room for a whole grid of codes.)
Playback: A dedicated machine with a 1280Hz high-speed camera films the screen, capturing the sequence as a video.
No keyboard macros. No sneaky uploads. Just photons and frames.
Once the video is extracted from the glass box:
Our parser watches the footage, frame by frame.
QR codes are detected and decoded in parallel.
Each chunk is sequence-tagged for ordering.
The resulting text is reassembled into a proper pg_dump.sql.
And just like that: the database lives again—fully exported with no digital transfer.
Only light and lenses.
QR Version: We used Version 40 QR codes (max capacity) with optimized binary mode for high density.
Error Correction: Level Q for resilience under compression/artifacts.
Screen Real Estate: 25×16 grid of codes per frame on a 1920×1080 pixel monitor—350 chunks per screen, around 1,016KB per frame.
Playback Rate: We achieved ~60 screens/sec = 21,000 chunks/sec, nearly 60MB/sec!.
Total Export Time: A full logical backup under 6GB was exported in less than 100 minutes!
This isn’t just a quirky story—it’s a reminder that PostgreSQL’s flexibility extends even into the absurd. Air-gapped systems aren’t rare in defense, finance, or critical infrastructure. And when normal tooling fails, PostgreSQL’s pluggable architecture gives you room to innovate, even in the tightest constraints.
We at Data Bene live for this kind of challenge. Whether it’s optimizing query plans or designing data exfiltration methods that look like spycraft, we’re here to help you make PostgreSQL dance—even when it’s stuck in a cage.
Want to try it yourself? Drop us a line—we love weird backups.
And if you’re thinking of streaming pg_restore into a laser light show, call us. We’re intrigued.
How often do companies express interest in open-source contribution? Clearly, by helping out in any way, the open-source project itself sees a benefit. But are there any advantages for the company that is giving back in any way? What are some contribution methods that a company can consider? These are all questions we hear about constantly—so let’s explore some of the answers discussed in this episode in a quick recap.
At Data Bene, we have a few customers that are interested in developing features or enhancements for the PostgreSQL ecosystem already.
These companies are interested in addressing bugs and adding new features that complement their use cases and tech stacks across PostgreSQL, Citus Data, and related technologies to accomplish two things:
Times change; the only way the upstream software will remain relevant, useful, and beneficial to the global audience using the product is if there are global contributions back to the same, ensuring it still meets real users needs from year to year.
Vendor lock-in is a huge problem in the software & services industry; giving back to open-source projects ensures that technology that is openly developed can continue to be so. Using FOSS technology means you avoid investing in a company that might close the code or restrict access, giving the end user freedom to continue using and developing essential tools that are part of their tech stack.
This kind of software is also subject to a highly visible development process, meaning it is much harder for privacy invasions, cybersecurity vulnerabilities, and more to be built into the underlying code.
Additionally, open-source software is built by individuals all over the world with a variety of perspectives and backgrounds; this ensures that it is thoroughly tested, with a wide range of features built-in that are actually useful to many end-users. This helps these kinds of projects to be successful for a number of years and continue to be so as long as there is a community willing to support each of them.
Case-in-point: PostgreSQL has been around for 35+ years of active development and is still topping developer surveys and charts today for being the most liked, most used, and most popular database solution—worldwide!
There are a few key ways to achieve this end-goal:
Cédric Villemain, Data Bene’s president, has developed pg_fincore and is currently working on StatsMgr, pg_psi, and other components that are designed to improve Postgres’ statistics capabilities.
Our team is also responsible for a number of contributions across projects like Citus Data and Zammad.
We make a point of sponsoring, presenting at, & advocating for PostgreSQL or open-source community conferences and user groups, such as PostgreSQL Europe, pgDay Paris, AlpOSS, Capitole du Libre, & more. Some of our team also individually have started or are on the organizational committees for various events such as the Barcelona & Madrid PostgreSQL User Groups and pgDay Lowlands. The impact of events on the larger project & community cannot be understated, and it is important to us to do all we can to contribute in this manner.
Finally, we help customers understand how to contribute to PostgreSQL and similar open-source projects. Through training, workshops, and collaboration, we encourage making meaningful contributions that fit their goals and support the greater community.
If you’re a developer who is interested in contributing to open-source and/or the PostgreSQL ecosystem, or helping customers with R&D requirements, our team is expanding—visit us at our website to see available positions!
Thinking about watching the full discussion? Check it out on YouTube:
We’ve finished our first round of episodes for Postgres Café as of this release! More episodes may or may not be pending… follow us on social media (like LinkedIn or Mastodon) to be updated on more to come. (Would you like to see more from this podcast series? Let us know!)
Subscribe to the playlist or check it out for interviews about open-source extensions like StatsMgr for efficient statistics management for PostgreSQL, an open-source change data capture (CDC) tool designed specifically for PostgreSQL called pgstream, & more. PostgreSQL is one of the most extensible databases on the market with a huge extension ecosystem; learn directly from the experts as you discover some of the options out there.
]]>The Citus database has experienced 127 releases since Mar 24, 2016 when it was first made freely open-source for open use and contributions. It’s a powerful tool that works natively with PostgreSQL, and seamlessly integrates with all Postgres tools and extensions. Continue reading for a summary of what we covered in this podcast episode!
So why does Citus Data exist, and what problems does it solve? Let’s delve into this by category.
Citus is designed to solve the distributed data modeling problem by providing methods in distributed data modeling to map workloads, such as sharding tables based on primary keys (especially useful for microservices and high-throughput workloads).
By distributing data across multiple nodes, you’re able to enable the horizontal scaling of PostgreSQL databases.
This allows developers to combine CPU, memory, storage, and I/O capacity across multiple machines for handling large datasets and high traffic workloads. It’s simple to add more worker nodes to the cluster and rebalance the shards as your data volume grows.
The distributed query engine in Citus is used to maximize efficiency, parallelizing queries and batching execution across multiple worker nodes.
Even in cases where there are thousands to millions of statements being executed per second, data ingestion is still optimized through finding the right shard placements, connecting to the appropriate worker nodes, and performing operations in parallel. All of this ensures high throughput and low latency for real-time data absorption.
Through the distributed data model, you can create redundant copies of tables and shard data across multiple nodes. Through this process, you can ensure the database remains resilient and available even when nodes crash, and maintain high availability as a result.
At Data Bene, our goal is to support forward momentum of upstream source code through ongoing development and code contributions. Cédric Villemain, among others on our team, constantly assesses for new feature additions or other improvements that can make a difference for users.
No matter whether you’re part of a DevOps team that is looking to build out distributed architecture for your PostgreSQL instances, or an end user such as a business analyst that is seeking efficient performance when handling vast amounts of data, Citus Data may be the perfect extension to support your use case.
If you have specific feature requests or concerns, our team here at Data Bene will help support you to contribute directly to Citus Data or can do so on your behalf to ensure the longevity of the project and relevance for your projects. Learn more about contributing to Citus Data by referencing the official CONTRIBUTING.md file.
Thinking about watching the full discussion? Check it out on YouTube:
More episodes are still being published for Postgres Café! Subscribe to the playlist for more interviews around open-source tools like StatsMgr for efficient statistics management for PostgreSQL, pgzx for the creation of PostgreSQL extensions using Zig, & more. Get ideas from the experts for new extensions to try out and maximize your Postgres deployments.
]]>In this episode, we introduce the just-released open source extension StatsMgr, created to continuously monitor and track events across PostgreSQL and the underlying system. Here’s a look at what this episode covered:
Originally the idea was to provide a simplified interface for metrics, while enhancing them with a wide variety of available types. This functionality was then expanded to address problems like:
This extension, in turn, is great at handling situations like when…
Currently, supported statistics include:
Each of these is registered with a handler that lets you fetch and manage these statistics, and also is accompanied by shared memory structures for storing historical snapshots.
Some of the next steps for the project will include adding in dynamic statistics such as pg_stat_user_tables, amongst others.
There are still many things to do, from subtle improvements to major new features. So of course there’s many opportunities to contribute to the project, no matter if you’re a new-comer or an advanced PostgreSQL developer. Interested in being a part of the effort? Check out CONTRIBUTING.md within the project.
For an in-depth exploration of StatsMgr and its capabilities, watch the full episode here:
We still have much more to come for Postgres Café. Subscribe to the playlist for episodes that feature more open-source tools like pgroll for zero-downtime schema migrations, Citus Data for distributed and scalable PostgreSQL as an extension, and more. Watch this space to learn how each tool can make working with Postgres smoother and more efficient.
]]>This article results from a code review I did for a customer.
Our customer created a pg_dump --schema-only of the target database to provide
me with the plpgsql code and database object structures to review. So far
so good.
I started to read the code and then became puzzled. The code looks like this:
CREATE FUNCTION xxx( p_id character, p_info character varying )
RETURNS integer
LANGUAGE plpgsql
AS $$
DECLARE
BEGIN
...
INSERT INTO t1
SELECT * FROM t2 WHERE t2.id = p_id;
...
END;
$$
;Maybe you saw nothing wrong with the function. Perhaps knowing the table
definition will help:
CREATE TABLE t2 (
id VARCHAR(130) NOT NULL
...
PRIMARY KEY (id)
);t2.id is always 130 characters long (in practice) and there are 400 million tuples.
So as you may have guessed, it seems odd to have the p_id CHARACTER matching id VARCHAR(130).
Moreover CHARACTER is the same as CHAR(1).
Our customer had not seen any issues with the code for years. Nevertheless, our customer told me that the function definition he wrote was not like that: it was meant to be p_id CHARACTER(130) - not CHARACTER.
So what went wrong? Let’s test around because it’s fun.
CREATE FUNCTION test( c character, d character varying )
RETURNS void
LANGUAGE plpgsql
AS $$
BEGIN
RAISE NOTICE 'c=%, d=%', c,d;
END;
$$;
SELECT test( '123465789', '987654321' );
NOTICE: c=123465789, d=987654321
test
------
(1 row)We have an interesting result here: no casting to CHAR(1) has been done.
Let’s see more details:
EXPLAIN (COSTS OFF,ANALYZE,VERBOSE)
SELECT test( '123465789', '987654321' );
NOTICE: c=123465789, d=987654321
QUERY PLAN
---------------------------------------------------------------------
Result (actual time=0.040..0.041 rows=1 loops=1)
Output: test('123465789'::bpchar, '987654321'::character varying)
Planning Time: 0.023 ms
Execution Time: 0.053 ms
(4 rows)We can see there was a cast to BPCHAR. As a reminder, BPCHAR is an alias of CHARACTER
and it can represent a string up to 10,485,760 characters.
Now let’s make another test:
CREATE FUNCTION test(c character(4))
RETURNS character(4)
LANGUAGE sql
AS $$
select c;
$$;As you can see, the language changed to SQL and the argument type and the return
type are CHAR(4). How does it execute?
SELECT test('123456789');
test
-----------
123456789
(1 row)
EXPLAIN VERBOSE SELECT test('123456789');
QUERY PLAN
-------------------------------------------
Result (cost=0.00..0.01 rows=1 width=32)
Output: '123456789'::bpchar
(2 rows)As you can see, even though you expect to process CHAR(4) data, you end up processing arbitrary length strings instead!!
However, do not rush to PostgreSQL mailing list to complain YET!
As a matter of fact, this behaviour is not a bug. The documentation states:
“The full SQL type syntax is allowed for declaring a function’s arguments and return value. However, parenthesized type modifiers (e.g., the precision field for type numeric) are discarded by CREATE FUNCTION. Thus for example CREATE FUNCTION foo (varchar(10)) … is exactly the same as CREATE FUNCTION foo (varchar) …”
This explains that CHARACTER(x) became CHARACTER aliased as BPCHAR. And as we saw, BPCHAR is not actually CHAR(1) but more like VARCHAR(10485760). This fully explains the behaviour.
Wait, wait , WAIT ! The original intention was to deal with CHAR(4) string - not any arbituary length strings.
Isn’t there any hope? No, sorry… (kidding.)
Reading the same documentation page, we see that “argtype” and “rettype” can be base, composite, or domain types, or can reference the type of a table column.
The trick is to create either a composite type or a domain to use as argtype or rettype.
Here are some examples:
-- Works in simple case trick
SELECT test( '12345789'::char(4) );
-- Domain trick
CREATE DOMAIN c4 AS char(4);
CREATE FUNCTION test(param c4)
RETURNS c4
AS $$
BEGIN
RAISE NOTICE 'param=%', param;
RETURN param;
END;
$$ LANGUAGE plpgsql;
SELECT test( '123456789' );
ERROR: value too long for type character(4)
SELECT test( '123456789'::char(4) );
NOTICE: param=1234
test
------
1234
(1 row)
SELECT test( '123456789'::c4);
NOTICE: param=1234
test
------
1234
(1 row)
SELECT pg_typeof( test( '123456789'::char(4) ) );
NOTICE: param=1234
pg_typeof
-----------
c4
(1 row)Now you should be happy with the result.
What? Not yet? Ok here is an additional trick.
-- Map a table structure
CREATE TABLE qq ( c char(4));
CREATE FUNCTION test(IN c qq, OUT d qq)
LANGUAGE sql
AS $$
SELECT c;
$$;
SELECT * FROM test(ROW('12345'));
ERROR: value too long for type character(4)
SELECT * from test(ROW('1234'));
c
------
1234Hmm, OK, but how is this is different from the domain trick?
-- Easy Type Alteration
ALTER TABLE qq ALTER c TYPE char(5);
SELECT * FROM test( ROW('12345') );
c
-------
12345Try to ALTER a domain - you will see how (not) easy it is.
The table definition trick allows for some flexibility as follows:
ALTER TABLE qq ADD ee int;
SELECT test( ROW('12345', 4) );
test
----------
(12345,4)
SELECT * FROM test( ROW('12345', 4) );
c | ee
-------+----
12345 | 4We hope you enjoyed this article and that you learnt something new and interesting!
]]>At Data Bene, we’re excited to unveil the latest version of our blog, where we will delve into PostgreSQL, open-source technologies, and the dynamic world of startups. With this update, we’re eager to bring you a curated selection of technical insights, industry news, and success stories to inspire and inform.
As loyal supporters of open-source solutions, we champion PostgreSQL for its ability to drive innovation and efficiency in modern businesses. Through our articles and tutorials, we aim to empower users to unlock it’s full potential. We will discuss our favourite features as well as provide updates on the latest advancements—topics we believe help in leveraging this powerful relational database management system.
In today’s rapidly evolving tech landscape, open source has become synonymous with collaboration, innovation, and community-driven development. We’re strongly committed to showcasing the transformative potential of open source as a result.
Because of this, in our blog we will explore:
With years of experience working alongside a diverse range of companies, from emerging startups to global enterprises, we’ve seen firsthand how innovative organizations leverage PostgreSQL to drive their success. Our blog will feature not only insights from entrepreneurs and tech leaders sharing their experiences but also in-depth articles from our own PostgreSQL experts. They will discuss their favorite PostgreSQL features and native tools, the challenges they’ve faced, and how they overcame them, providing a unique, technical perspective on overcoming real-world challenges.
We’ll also dive into emerging trends and new technologies that are shaping the future of PostgreSQL, offering a window into how startups and established businesses alike are navigating the fast-evolving landscape of data management. Whether it’s through interviews with founders or analysis of groundbreaking projects, we’ll highlight how PostgreSQL continues to be a key enabler of innovation in today’s tech-driven world.
Finally, at Data Bene, sustainability isn’t just a buzzword—it’s a core value that guides our every endeavor. As we embark on new development initiatives and explore cutting-edge technologies, we’re committed to minimizing our environmental footprint and promoting eco-friendly practices. From optimizing software efficiency to reducing energy consumption, we’re dedicated to building a more sustainable future, one line of code at a time. Consequently, some of the articles we release will highlight these practices and open up the topic of sustainability within PostgreSQL development practices for discussion.
We invite you to join us on this exciting journey of exploration, discovery, and innovation. Whether you’re a PostgreSQL enthusiast, an open-source advocate, a startup aficionado, a long-time developer, or just curious, there’s something for everyone in our blog! So bookmark this page, follow us on LinkedIn and keep an eye out for our upcoming newsletter. Anywhere you follow, you’ll be able to stay tuned for regular updates packed with valuable insights and inspiration.
Thank you for being a part of our community. Together, let’s embark on a journey of growth, learning, and shared success.
]]>In the morning, Frédéric Delacourt will present “Everything you always wanted to know about Connection Pools (but were afraid to ask)”, from 9am to 12:15pm.
In the afternoon, it will be Cédric Villemain’s turn with “CREATE EXTENSION, yours preferably”, from 1:45pm to 5pm.
We are delighted to welcome you there. You will find the details of the training on the agenda of the 2 days, and you can register on the registration page.
We hope to see many of you there!
The Data Bene team.
]]>We will host a conference on Citus Data: Distributed database without complex.
]]>