pgsql-hackers/digest / threads mailing list source generated · LLM-summarized

The pgsql-hackers
Weekly Digest

A reader's guide to the discussions, patches, and bickering on the primary development mailing list of PostgreSQL. Summaries are produced by a generative model — useful for orientation, not for citation.

Friday, November 20, 2026
50 hot threads
Note Thread statuses and summaries are generated by an LLM-based system and may contain inaccuracies. Always defer to the linked archive thread before quoting.

Hot Threads

showing 50 of 50 threads
01 index prefetching Patch Review 99 msgs Nov 19, 23:29
opened Feb 18, 04:21 ·last activity Nov 19, 23:29

This extensive thread details the development and refinement of index prefetching, including architectural changes and bug fixes. Early discussions focused on testing methodologies for varying storage latencies and measuring 'unconsumed IO.' The proposer later raised a concern about an incomplete GiST cleanup-lock fix on standbys, comparing it to a known nbtree issue. Subsequent patch versions (v36-v38) introduced a significant redesign of hashgetbatch for simpler sibling traversal, and an experimental patch for read stream request deduplication to resolve performance regressions caused by redundant block requests. The proposer also addressed performance inconsistencies under `io_method=sync` by suppressing `posix_fadvise` hints in that mode. Reviewer 1 provided detailed feedback on `tableam_index_pos_advance` logic, documentation, and a potential off-by-one error. Commenter 1 reported a segfault bug in SP-GiST index-only scans on released branches and proposed a back-portable fix. Commenter 2 offered substantial architectural feedback regarding the table AM interface, highlighting potential blocking issues for out-of-tree AMs due to new batch callbacks and per-item recheck requirements, and offered to contribute generic implementations.

Recent reply
Nov 19, 23:01 The latest reply from the proposer, containing v39 of the patch series, primarily serves to ensure the extensive set of patches continues to apply cleanly. This indicates ongoing development, re-basing, and integration work for the index prefetching feature and its related changes.
archive ↗
02 Snapshot related assert failure on skink Proposed 16 msgs Nov 20, 08:29
opened Mar 4, 22:25 ·last activity Nov 20, 08:29

The thread begins with a report of a snapshot-related assert failure on the buildfarm. The initial investigation by the proposer and a reviewer identified a bug where xactCompletionCount was not correctly updated during hot standby, leading to snapshot reuse with an older xmin. An experimental fix successfully prevented the initial failures, and a committer stated they would prepare and commit a patch. However, the latest message in the thread reports that the assertion can still be triggered via a different path, even after the initial fix was committed. This new reproduction mechanism, involving ProcArrayApplyRecoveryInfo() and `latestObservedXid`, is described, along with a proposed fix to address it.

Recent reply
Nov 20, 08:25 The latest commenter reports that despite a previous fix for a snapshot-related assertion, the issue can still be triggered via a new path on standby. The commenter describes a mechanism involving ProcArrayApplyRecoveryInfo() and `latestObservedXid` not being updated, leading to an incorrect snapshot xmin. A proposed fix suggests updating `latestObservedXid` within ProcArrayApplyRecoveryInfo() to align with `latestCompletedXid`.
archive ↗
03 Use WALReadFromBuffers in more places Patch Review 26 msgs Nov 20, 02:29
opened Sep 14, 05:04 ·last activity Nov 20, 02:29

This long-running thread aims to enhance WAL reading efficiency by enabling `WALReadFromBuffers` in more contexts, particularly for walsenders and the WAL summarizer, allowing them to opportunistically read WAL directly from memory buffers instead of files. The initial patches focused on implementing this optimization. Benchmarking showed significant reductions in disk I/O for walsenders, especially those closely following the publisher. A key challenge arose when tests expecting disk reads started failing due to all WAL being served from buffers; this led to the introduction of `IOOP_HIT` tracking in `pg_stat_io` to distinguish buffer hits from file reads. Discussions also refined how these "hits" are counted (per page rather than per function call) and addressed potential issues with `WAL` segment tracking (`ws_tli`) when file reads are bypassed. One patch (0001) was committed, and subsequent patches incorporate fixes and refinements for the remaining components.

Recent reply
Nov 20, 01:49 The proposer submitted v11, addressing a reported issue where `state->seg.ws_tli` was not being updated when WAL was entirely read from buffers, bypassing file I/O. This fix ensures correct `WAL` segment tracking and error reporting, upholding the `XLogPageReadCB` callback rule. This is part of the ongoing review for the remaining patches in the series.
archive ↗
04 Disable startup progress timeout during standby WAL replay Patch Review 12 msgs Nov 20, 07:28
opened Jun 11, 00:13 ·last activity Nov 20, 07:28

This thread addresses an inefficiency in PostgreSQL's startup process where the startup progress timeout remains active during standby WAL replay, leading to unnecessary periodic wakeups. The original intent, as described in code comments, was to suppress recovery progress logging in standby mode to avoid log bloat. The proposer's patch aims to explicitly disable this timeout before entering WAL replay in standby mode, ensuring that early startup operations still report progress while preventing superfluous activity during extended replay. Discussions focused on the optimal placement for the `disable_startup_progress_timeout()` call, considering various recovery paths and how other functions like `ResetUnloggedRelations()` might re-enable the timeout. An unrelated issue regarding `disable_all_timeouts(false)` was identified and subsequently split into a separate thread for independent review. The latest patch version includes clarified comments, a note about standby progress during initial crash recovery, and a fix for a minor bug where changing `log_startup_progress_interval` to zero during initial crash recovery might not properly stop the timer.

Recent reply
Nov 20, 06:43 The proposer provided updated patches, clarifying code comments and adding a note about standby recovery progress during initial crash recovery. Additionally, the proposer introduced a fix in a separate patch (0001) to address a minor bug: `disable_startup_progress_timeout()` could fail to stop an active timer if `log_startup_progress_interval` is set to zero during initial crash recovery, with a recommendation for backpatching.
archive ↗
05 [RFC] Lock-free XLog Reservation from WAL Patch Review 48 msgs Nov 19, 15:29
opened Jan 2, 07:05 ·last activity Nov 19, 15:29

This thread discusses the "Lock-free XLog Reservation from WAL" to reduce lock contention during WAL insertions on systems with many virtual CPUs. The initial proposal outlined a multi-faceted approach, including removing `PrevBytePos` for atomic `CurrBytePos` updates, updating the prev-link of the next XLog, and breaking sequential write convention, along with increasing `NUM_XLOGINSERT_LOCKS`. Early patch versions and performance tests, particularly when `NUM_XLOGINSERT_LOCKS` remained at the default 8, sometimes showed performance decreases or mixed results, indicating that the new complexity might not always outperform the existing spinlock. The patch was iterated on to fix issues like a null pointer dereference on Windows. Much later, the proposer submitted v8 of a CAS128 patch, which aims to update `CurrBytePos` and `PrevBytePos` atomically, eliminating the need for a complex predecessor hash table. This v8 patch, which includes fallback to spinlocks if native CAS128 is unavailable, showed significant performance gains (up to +40.4% with 64 locks) on high-core count Intel Xeon systems for HammerDB TPROC-C workloads. The proposer noted that increasing locks might still regress a specific `pg_logical_emit_message` workload.

Recent reply
Nov 19, 15:21 The latest message introduces v8 of the CAS128 patch for lock-free XLog reservation. It proposes updating `CurrBytePos` and `PrevBytePos` simultaneously using native CAS128 (with spinlock fallback), eliminating the need for a predecessor hash table. Benchmarks on Intel Xeon CPUs show substantial performance gains (up to +40.4% with 64 insertion locks) in HammerDB TPROC-C workloads. The proposer also notes that a separate GUC patch for `wal_insert_locks` is being submitted.
archive ↗
06 A bug in recovery prefetcher, triggering asserts / FATAL: checkpoint request failed Committed 11 msgs Nov 20, 10:28
opened Oct 25, 23:37 ·last activity Nov 20, 10:28

The proposer identified and fixed a pre-existing bug in the recovery prefetcher, present since PG 15. The bug caused XLogPrefetcherBeginRead to not reset prefetcher->record, leading to XLogPrefetcherReadRecord being confused and triggering assert failures in development builds or "FATAL: checkpoint request failed" errors in regular builds, particularly with reduced maintenance_io_concurrency. The fix involved simply resetting prefetcher->record to NULL. The proposer provided a patch with the fix and a new TAP test to reproduce the issue. Reviewers confirmed the bug and the effectiveness of the fix, with one reviewer suggesting an additional check in FinishWalRecovery() as a separate improvement. The proposer committed and backported the fix, including the TAP test. However, buildfarm members later reported failures with the new test on platforms lacking posix_fadvise(), as these platforms reject non-zero maintenance_io_concurrency. The discussion then shifted to how to correctly skip the test on such unsupported platforms, particularly considering differences in prefetching support across PostgreSQL versions and operating systems like macOS. The proposer plans to add conditional skipping for older versions.

3 recent replies
Nov 20, 09:46 The latest reply from the proposer clarifies the strategy for handling the buildfarm failures for the newly committed recovery prefetcher test. The proposer intends to add a skip condition for the test only on PostgreSQL versions 15 through 17. This approach aims to allow the test to run universally on version 18 and later, even on macOS, assuming the platform-specific prefetch for macOS in v18+ makes maintenance_io_concurrency = 1 acceptable there.
Nov 20, 02:44 Commenter 4 points out a potential issue with the proposed method for skipping the new recovery prefetcher test on unsupported platforms. The `check_pg_config("#define HAVE_POSIX_FADVISE 1")` condition might incorrectly skip the test on macOS for v18 and later, as these versions have platform-specific prefetch support despite not using `posix_fadvise()`, meaning the `maintenance_io_concurrency` setting would still be valid.
Nov 20, 01:54 The proposer provided a solution to a recent buildfarm failure related to the newly committed recovery prefetcher test. The test, which relies on `posix_fadvise()`, failed on platforms lacking this functionality when `maintenance_io_concurrency` was set. The proposer detailed a plan to use `check_pg_config` to conditionally skip the test on unsupported platforms and stated they would push this fix soon.
archive ↗
07 Assert "target_wal_endrec == divergerec" in pg_rewind Committed 6 msgs Nov 20, 08:29
opened Nov 16, 19:13 ·last activity Nov 20, 08:29

The thread discusses an Assert("target_wal_endrec == divergerec") failure in pg_rewind that occurs when the target server is a direct ancestor of the source and stopped earlier on the common timeline. This assertion was previously identified as bogus. A patch was proposed to remove this assertion and include a TAP test for the scenario. Reviewers confirmed the assertion is indeed spurious and agreed that removing it is correct. The test case, initially integrated in a non-standard way, was refined and moved to a new dedicated file to cover important cases of direct ancestors based on backups. The removal of the assertion and the addition of the test are now slated for commitment.

3 recent replies
Nov 20, 08:15 The committer confirms the decision to remove the identified assertion, agreeing with previous evaluations that it is bogus. The committer also acknowledges the value of the included TAP test, noting it provides useful coverage for scenarios involving direct ancestors based on backups, which was previously lacking.
Nov 19, 19:47 The latest reply from a reviewer confirms that the assertion discussed in the thread is indeed bogus and that the proposed patch to remove it is worth committing. However, the reviewer notes that the accompanying TAP test, intended to cover the issue, deviates from the standard `pg_rewind` testing model and suggests it should be brought in line with the existing test infrastructure.
Nov 16, 19:13 The proposer submits a rebased patch to remove a "bogus" assertion (`Assert(target_wal_endrec == divergerec)`) in `pg_rewind`. This assertion causes crashes in assertion builds when the target is a direct ancestor of the source. The patch, based on a previously suggested fix, includes TAP coverage and does not alter behavior in non-assertion builds. The proposer is seeking confirmation on its commitment.
archive ↗
08 [BUG] Take a long time to reach consistent after pg_rewind Patch Review 16 msgs Nov 20, 08:29
opened Apr 10, 09:57 ·last activity Nov 20, 08:29

The thread discusses a bug where a server, after pg_rewind, can take an unboundedly long time to reach a consistent state if the source server is idle. This occurs because pg_rewind sets minRecoveryPoint using pg_current_wal_insert_lsn(), which can point past a WAL page header where no actual WAL record ends, making the LSN unreachable without further WAL activity. Several solutions were discussed: adjusting minRecoveryPoint to the beginning of the page, or using GetXLogInsertEndRecPtr() or the flush LSN. The consensus shifted towards using the flush LSN due to its crash-safety properties. The latest patch (v5) implements this, updates documentation, and passed review for correctness and backpatching suitability, though a regression test is still considered.

3 recent replies
Nov 20, 08:19 The latest commenter expresses support for backpatching the v5 patch and suggests a regression test for the issue in master. The commenter also raises concerns about the "extra header skipping" behavior of pg_current_wal_insert_lsn()/GetXLogInsertRecPtr() and whether further documentation on this function's intricacy would be beneficial.
Nov 20, 00:34 The latest reply (reviewer 1) confirms that the updated patch from the committer, which includes documentation fixes and cosmetic changes, looks good. The reviewer also agrees that the fix should be backpatched to all supported PostgreSQL versions, indicating that the patch is ready for widespread application.
Nov 19, 17:46 A commenter provides an updated version of the patch, which includes fixes to `pg_rewind`'s documentation to correctly refer to `pg_current_wal_flush_lsn()` as the minimum consistency LSN for a live primary. The commenter also refined the commit message and made other minor cosmetic adjustments. The commenter explicitly asks whether the fix should be applied only to the master branch or if it warrants backpatching to all supported PostgreSQL versions, providing backpatch versions for consideration.
archive ↗
09 Don't use the deprecated and insecure PQcancel in our frontend tools anymore Committed 33 msgs Nov 19, 14:30
opened Dec 14, 14:40 ·last activity Nov 19, 14:30

This thread addresses a security concern by replacing the insecure PQcancel function with newer, encryption-aware cancellation APIs in various PostgreSQL frontend tools like psql. The core change involves refactoring the cancellation logic to use a dedicated thread for sending cancel requests, achieving consistency across platforms. Initial discussions included the feasibility of backporting this complex change to recent PostgreSQL versions. During review, a subtle behavioral change was noted regarding psql's Ctrl+C handling in script mode, particularly during \copy operations. The proposer clarified that the new behavior, where Ctrl+C terminates the entire script, was an intentional fix for what was considered a bug. After addressing minor issues, including pager display, the first two patches in the series, which focused on this refactoring and behavior correction, were committed.

Recent reply
Nov 19, 14:16 The reviewer announces that the second patch, titled "psql: Replace cancel_pressed with CancelRequested," has been committed, following the prior commitment of the first patch in the series.
archive ↗
10 Hash aggregate collisions cause excessive spilling Patch Review 17 msgs Nov 20, 04:28
opened Feb 19, 14:49 ·last activity Nov 20, 04:28

The thread addresses a critical performance bug in HashAggregate nodes where hash collisions cause excessive spilling, leading to queries getting stuck indefinitely. The initial problem was traced to aggstate->hash_metacxt exceeding work_mem, forcing frequent spills with an n² overhead. The proposer initially suggested a fix using parent->plan->plan_node_id as a hash initialization vector (hash_iv) to improve hash distribution. This solution, iterated upon with suggestions from reviewers, now uses hash_combine with murmurhash32 on the parallel worker number and plan node ID, and includes comments for ExecInitSubPlan. Reviewers confirmed the fix's effectiveness but also highlighted underlying issues like poor n_distinct statistics and peculiar hash function behavior. The discussion has evolved to also include improvements to hash_agg_check_limits() to prevent inefficient hash table resizing before spilling and better memory management for grouping sets. The issue is considered urgent, with suggestions for backpatching.

3 recent replies
Nov 20, 03:33 The latest reply from reviewer 2 focuses on optimizing the hash calculation path (ExecJustHashInnerVarWithIV()) and proposes a critical fix for hash_agg_check_limits() to prevent inefficient spilling. They suggest modifying the spilling condition to account for memory usage more precisely, such as requiring (entry_mem + tval_mem) to exceed a fraction of hash_mem_limit. Additionally, they suggest re-evaluating initial hash table bucket sizing to avoid resource crowding and recommend benchmarking the proposed changes.
Nov 20, 00:47 The latest reply (commenter 1) discusses strategies to prevent hash table crowding and excessive spilling. They suggest avoiding doubling the hash table size right before spilling, possibly by limiting new group insertions if they would cause growth. Another idea is to optimize grouping set processing by destroying hash tables of completed sets to free memory for active ones.
Nov 19, 20:01 The latest reply from a commenter highlights the rediscovery of the original issue in TPC-H scale 10 workloads, where a query became unresponsive for over an hour. This re-emphasizes the severity and ongoing relevance of the problem, particularly in parallel hash aggregates. The commenter suggests that recent changes make the fix more urgent and recommends backporting it to earlier PostgreSQL versions.
archive ↗
11 Report buffer access strategies in EXPLAIN Patch Review 23 msgs Nov 20, 09:28
opened Oct 27, 18:56 ·last activity Nov 20, 09:28

The proposer initiated a patch series to enhance `EXPLAIN (ANALYZE, BUFFERS)` by reporting buffer access strategies (BAS), including whether normal buffer replacement or a ring buffer strategy is used. This builds on previous discussions and `pg_stat_io` already exposing similar data. Initial feedback from reviewer 1 included suggestions for documentation wording, identified an issue with `SampleScan` in parallel plans not reporting strategies for worker-executed scans, and raised concerns about assumptions in index scan instrumentation. A commenter questioned the consistency of terminology like "strategy ring". The proposer submitted v5 and v6 patches, incorporating some feedback and performing mechanical rebases. The discussion is ongoing regarding the precise wording for L2 cache descriptions in the documentation and whether specific strategy names should be explicitly listed or if a glossary link is sufficient, with reviewer 2 expressing reservations about some additions in patch 0003.

3 recent replies
Nov 20, 09:27 Reviewer 2, in response to the proposer's query, reiterated their reservations regarding specific additions in patch 0003. They maintained that explicit listing of strategy names in the documentation might be unnecessary if a glossary link is provided, even considering the potential for backpatching to stable branches.
Nov 20, 08:10 The latest commenter responds to a reviewer's suggestion to simplify documentation by merging patch 0003 into 0002. The commenter asks if the reviewer missed their previous reply regarding this suggestion, indicating there might be an unaddressed point or a need for further discussion on the proposed structural change to the patch.
Nov 19, 11:59 The latest reply from the proposer introduces v6 of the patch series. This version is a mandatory mechanical rebase, necessitated by a recent commit (556c025db96). This rebase ensures that the patch series continues to apply cleanly against the current master branch, reflecting the ongoing iterative refinement process based on reviewer feedback.
archive ↗
12 let ALTER TABLE DROP COLUMN drop whole-row referenced object Discussing 18 msgs Nov 19, 17:28
opened Sep 8, 02:53 ·last activity Nov 19, 17:28

This thread discusses modifying `ALTER TABLE DROP COLUMN` to correctly handle objects (constraints, indexes, policies, triggers) that contain whole-row references. The proposer initially identified that such objects are not dropped automatically, leading to inconsistencies, contrary to documentation. An early patch aimed to rectify this by automatically dropping these whole-row dependent objects when a column is dropped and making `ALTER COLUMN SET DATA TYPE` error out if such dependencies exist. Reviews led to patch refinements, addressing implementation details and memory leaks. A reviewer confirmed the patch's correct behavior for various scenarios, including `CASCADE` and `ALTER COLUMN TYPE` errors. However, a commenter later raised a fundamental design concern, suggesting that the proposed scanning of system catalogs for whole-row references is inefficient. This commenter proposed an alternative approach involving registering whole-row `Vars` in `pg_depends` for proper invalidation and argued that whole-row indexes should be rebuilt rather than dropped when a column is modified. The latest reply further highlighted issues with silently dropping unique indexes and `CHECK` constraints, providing examples of data integrity problems and advocating for rebuilding or rechecking these objects.

Recent reply
Nov 19, 17:00 The latest commenter provided concrete examples illustrating how the current patch's behavior of silently dropping whole-row unique indexes and `CHECK` constraints during a column drop can lead to data integrity issues. The commenter argued that rebuilding or rechecking these objects is preferable, as they still apply to remaining columns, and quietly dropping them removes existing data guarantees. The commenter then contrasted the current patch's approach with an alternative that would either error out or rebuild non-unique indexes.
archive ↗
13 Table AM callback for logical decoding support (REPACK CONCURRENTLY) Patch Review 4 msgs Nov 20, 09:28
opened Nov 19, 16:34 ·last activity Nov 20, 09:28

The proposer identified a limitation where `REPACK (CONCURRENTLY)` is restricted to heap tables because the `TableAmRoutine` lacks a clear way to indicate if a Table Access Method (AM) supports logical decoding, potentially leading to silent data loss. The proposer proposed adding an optional `relation_supports_logical_decoding` callback to `TableAmRoutine` to address this. This would be the second such addition for out-of-tree AMs. The patch includes a test module to demonstrate its functionality. Commenter 1 supported the AM method approach but questioned the necessity of a `Relation` argument for the callback, as logical decoding support seems AM-level. Commenter 2 suggested an alternative of providing the AM's `RmgrId` to check for an `rm_decode()` callback. Commenter 3 raised a testing concern about `wal_level=minimal` and suggested setting `wal_level=replica` for tests.

3 recent replies
Nov 20, 09:09 Commenter 3 brought up a testing consideration for the patch. They noted that `REPACK(CONCURRENTLY)` statements failed `wal-level` checks when `wal_level` was set to `minimal`. They suggested setting `wal_level = replica` in a temporary configuration for `make` and `meson` to properly accommodate these tests, drawing a parallel with `pg_walinspect` practices.
Nov 20, 07:54 The latest commenter suggests an alternative API design, proposing to add a callback or field that provides the Table AM's RmgrId. This would allow REPACK (CONCURRENTLY) to check for the existence of the rm_decode() callback, thereby determining logical decoding support, rather than relying on a separate relation_supports_logical_decoding callback.
Nov 20, 02:40 A commenter agrees with the general concept of using a Table AM method for logical decoding support. However, they raise a question regarding the design of the proposed `relation_supports_logical_decoding` callback, specifically asking why it requires a `Relation` argument, as logical decoding capabilities are typically an AM-level attribute rather than being dependent on individual relations.
archive ↗
14 Parallel GIN, BRIN builds fail to maintain ii_BrokenHotChain Patch Review 4 msgs Nov 20, 10:28
opened Nov 20, 00:46 ·last activity Nov 20, 10:28

The proposer identified a bug in parallel GIN and BRIN index builds, where they fail to correctly maintain the ii_BrokenHotChain flag. This oversight means that if an individual parallel worker sets ii_BrokenHotChain, the information is not propagated to the leader process, leading to pg_index.indcheckxmin not being set correctly. Consequently, older snapshots might incorrectly use the index, risking inconsistent results. The proposer provided an isolation test, generated by an LLM, that demonstrates this incorrect behavior. Reviewer 1 acknowledged the issue and offered to fix it. A commenter then indicated they had encountered the exact same problem, had already developed a fix as part of a larger refactoring to prevent future similar issues, and confirmed that the proposer's test passed with their patch. Reviewer 1 confirmed they would use the provided fix.

3 recent replies
Nov 20, 10:04 The latest reply from reviewer 1 thanks the commenter for providing a fix that also addresses a broader refactoring. Reviewer 1 confirms that they will use this provided patch for the bug fix, acknowledging its helpfulness in resolving the ii_BrokenHotChain issue during parallel GIN and BRIN index builds.
Nov 20, 02:54 A commenter contributed a v2 patch, based on their previous work, that specifically addresses the reported bug. This patch, combined with the proposer's isolation test, was confirmed to resolve the `ii_BrokenHotChain` propagation issue in parallel GIN and BRIN index builds, allowing the test to pass successfully.
Nov 20, 01:55 A reviewer acknowledged the bug report and the provided reproducer test. The reviewer confirmed the issue, which affects parallel GIN and BRIN index builds' ability to correctly maintain the `ii_BrokenHotChain` flag, and indicated that they would take responsibility for fixing it.
archive ↗
15 [PATCH] Add planner support function to two-argument regexp_like() Patch Review 6 msgs Nov 20, 01:28
opened Jul 29, 04:11 ·last activity Nov 20, 01:28

The proposer identified that regexp_like() (two-argument form) lacks a planner support function, unlike the ~ operator, leading to inefficient sequential scans instead of index scans for regular expressions with fixed prefixes. The initial patch aimed to link regexp_like() to the existing textregexeq_support() function, which can extract prefixes for B-tree index conditions. An early reviewer confirmed installcheck-world passed. The proposer explained the mechanism to another reviewer. A reviewer suggested unnecessary regression tests could be removed, which the proposer agreed to. However, a later reviewer found a regression in row-count estimation when expression index statistics are available for regexp_like(), causing the support function to return a default selectivity instead of using the statistics, leading to underestimates. The reviewer suggested addressing this problem.

Recent reply
Nov 20, 01:21 The latest reply (commenter 3) reports a new regression caused by the patch: it introduces incorrect row-count estimation for regexp_like() when expression index statistics are available. The support function returns a default selectivity, overriding more accurate statistics and leading to significant underestimates. The commenter suggests addressing this alongside existing similar issues.
archive ↗
16 psql: avoid over-reading unterminated prompt escapes Committed 7 msgs Nov 20, 09:28
opened Sep 21, 03:30 ·last activity Nov 20, 09:28

The proposer reported a `psql` crash, specifically a heap-buffer-overflow, observed with AddressSanitizer when `PROMPT1` was set to an unterminated variable like `%:aa`. This issue stemmed from `psql` attempting to over-read characters beyond the allocated buffer for prompt escapes. After an initial patch submission, reviewer 1 suggested adding a TAP test for custom prompts, and commenter 1 provided a detailed review, confirming the fix's correctness across different versions and with ASan, and also supplied an optional TAP test. The committer decided against including the optional TAP test due to its unreliable bug detection without ASan and the associated maintenance overhead. Subsequently, the committer announced that the patch had been pushed, resolving the issue. The proposer expressed thanks for the commit.

3 recent replies
Nov 20, 07:34 The proposer conveyed gratitude to the committer for integrating the patch. This final message confirms the successful commit of the solution to the reported heap-buffer-overflow issue in `psql`, which occurred when processing malformed prompt escape sequences.
Nov 20, 06:16 The committer's latest reply confirms that the `0001` patch, which resolves the `psql` over-reading bug with unterminated prompt escapes, has been successfully pushed to the codebase, concluding the thread and committing the fix.
Nov 19, 10:59 In the latest reply, reviewer 2 confirmed that the v2 patch correctly addresses the heap-buffer-overflow bug. They also stated their intention to commit the patch, but recommended against including the proposed optional TAP test. The decision to omit the test was based on its limited utility in a standard build (as it only reliably caught the bug with AddressSanitizer) and the desire to avoid unnecessary maintenance overhead.
archive ↗
17 EXPLAIN ANALYZE CREATE TABLE AS EXECUTE runs non-SELECT prepared statements Committed 4 msgs Nov 20, 05:29
opened Nov 19, 09:51 ·last activity Nov 20, 05:29

The proposer identified a bug where EXPLAIN ANALYZE CREATE TABLE AS EXECUTE would incorrectly execute non-SELECT prepared statements, unlike CREATE TABLE AS EXECUTE which correctly rejects them. This could lead to unexpected data modifications and table creation with rows from non-SELECT statements (like INSERT ... RETURNING). The proposed patch adds the necessary check to ExplainExecuteQuery, aligning its behavior with CREATE TABLE AS EXECUTE. Reviewers confirmed the issue and the effectiveness of the patch, noting it's a straightforward fix that should be backpatched. Discussions also touched upon related but out-of-scope issues regarding EXPLAIN behavior with existing tables and volatile functions. The general consensus is that the patch correctly closes a critical EXPLAIN loophole.

3 recent replies
Nov 20, 05:00 The reviewer 3 confirmed the bug is an oversight in a previous commit and that the attached patch provides a straightforward and effective fix. They expressed approval for the proposed solution.
Nov 20, 04:02 The latest reply from reviewer 2 confirms the bug on the current master and validates that the proposed patch correctly adds the SELECT statement check to ExplainExecuteQuery. They verified that the EXPLAIN ANALYZE command now fails with the expected "prepared statement is not a SELECT" error, preventing unintended data modifications and table creation. The patch was applied and passed all regression tests, indicating it correctly closes the EXPLAIN hole.
Nov 19, 09:51 The latest message, which also initiates the thread, reports a bug where `EXPLAIN ANALYZE CREATE TABLE AS EXECUTE` incorrectly executes non-`SELECT` prepared statements. The proposer provides a patch to add the necessary validation, ensuring `EXPLAIN ANALYZE` adheres to the same `SELECT`-only restriction as `CREATE TABLE AS EXECUTE`, and suggests the fix for backpatching due to its small size and clear correction.
archive ↗
18 Checkpoints stuck after a failed bulk write inside a savepoint Discussing 5 msgs Nov 20, 08:29
opened Nov 3, 05:02 ·last activity Nov 20, 08:29

The thread addresses a bug where checkpoints could get stuck indefinitely after a bulk write operation fails within a savepoint. This happens because the DELAY_CHKPT_START flag is set outside a critical section and not cleared upon error, causing subsequent checkpoints to wait. The proposer's patch aims to eliminate the need for this flag by reordering operations: first registering the sync request, then checking the redo pointer, and performing an immediate fsync only if needed. A reviewer confirmed the fix and its underlying logic. The discussion then focused on a minor performance implication of the proposed fix, involving a potentially redundant fsync. An alternative approach was suggested to further minimize this overhead. The necessity of backpatching was also questioned.

3 recent replies
Nov 20, 08:00 The latest commenter clarifies a typo in their previous message, correcting "smgrregistersync() and the smgrregistersync() calls" to "smgrregistersync() and the smgrimmedsync() calls" when discussing the window for useless syncs in the alternative solution.
Nov 16, 17:20 The proposer clarified a point raised by the reviewer regarding the potential for an extra `fsync` within a checkpoint. The proposer confirmed that this overhead, involving one additional `fsync` per segment on an already flushed file, would only materialize if a checkpoint were to run concurrently during the bulk write operation.
Nov 16, 11:39 A reviewer provided positive feedback on the v1-0001 patch, confirming it fixes the reported issue. The reviewer noted that the previous problem, where the test would hang due to checkpoints not finishing, is resolved with the patch applied. The reviewer also approved of the solution's idea of reordering operations to remove the problematic flag, ensuring that nothing leaks when an error is caught by a savepoint.
archive ↗
19 [PATCH] Reject duplicate SQL/JSON PASSING argument names Patch Review 6 msgs Nov 20, 09:28
opened Nov 18, 02:55 ·last activity Nov 20, 09:28

The proposer submitted a patch to enforce uniqueness for variable names in SQL/JSON `PASSING` clauses. Previously, duplicate names were silently handled by using the first matching value, which could lead to confusion. The patch introduces an error (`ERRCODE_DUPLICATE_ALIAS`) when duplicate `PASSING` argument names are detected in `JSON_EXISTS`, `JSON_QUERY`, `JSON_VALUE`, and `JSON_TABLE`, aligning behavior with Oracle and promoting clearer error reporting. Reviewer 1 provided comprehensive feedback, suggesting improvements to code style, documentation, test coverage, and the commit message. The proposer incorporated all the suggested changes into a v2 patch, which Reviewer 1 subsequently reviewed and marked as "ready for committer."

3 recent replies
Nov 20, 07:38 Reviewer 1 confirmed that the updated patch, incorporating all previous feedback, has been marked as "ready for committer." This indicates that the patch has successfully completed its review phase and is awaiting final integration into the codebase.
Nov 20, 05:59 The latest reply from the proposer confirms that the v2 patch, which rejects duplicate SQL/JSON `PASSING` argument names, has been successfully registered in the current CommitFest. This follows its approval by a reviewer after incorporating all suggested improvements, including code style, documentation updates, and extended test coverage.
Nov 18, 23:51 The reviewer confirmed that the v2 patch addresses all previously raised comments and looks good. The updates included reordering the duplicate name check, adopting a modern `ereport()` style, adding documentation to `func-json.sgml` stating the uniqueness requirement for `PASSING` variable names, expanding regression tests to cover `JSON_EXISTS` and `JSON_VALUE`, and updating the commit message to list all affected functions.
archive ↗
20 pgstat: Flush some statistics within running transactions, take 2 Patch Review 53 msgs Nov 20, 08:29
opened May 17, 14:34 ·last activity Nov 20, 08:29

The thread proposes a mechanism to flush some PostgreSQL statistics within running transactions, rather than only at transaction boundaries. This aims to provide more up-to-date metrics, especially for long-running transactions. The proposer introduced SQL and C APIs to trigger on-demand flushes. The discussion involved refining which counters are flushed mid-transaction versus deferred until commit/rollback, particularly for transactional write counters like tuple inserts/updates/deletes. Subsequent patch versions addressed issues found during review, including correct handling of TRUNCATE within procedures and interactions with ANALYZE, and the behavior of CHECK_FOR_INTERRUPTS in the flush loop. A recently identified n_ins_since_vacuum issue was determined to be a pre-existing bug not introduced by the patch.

3 recent replies
Nov 20, 08:17 The latest commenter agrees with the proposer that the recently identified n_ins_since_vacuum issue is a pre-existing bug, not caused by the current patch, and confirms it has been present since a specific earlier commit. The commenter indicates they will initiate a dedicated thread to discuss and address this unrelated bug.
Nov 20, 06:58 The latest reply from commenter 2 questions whether a particular `n_ins_since_vacuum` reporting issue, which the proposer identified as a pre-existing bug on HEAD, might have been inadvertently introduced or influenced by a recent change made by commenter 2. This suggests ongoing investigation into the interaction of the proposed patch with existing codebase behavior and responsibilities for identified issues.
Nov 17, 14:08 The proposer responded to a newly reported bug concerning an incorrect `n_ins_since_vacuum` value after a sequence involving an aborted savepoint, a forced flush, `TRUNCATE`, and `COMMIT`. The proposer stated that this particular issue is present in the current master branch and is not introduced by the patch. Therefore, the proposer suggested that this specific problem should be addressed in a separate mailing list thread.
archive ↗
21 [PATCH v1] Preserve implicit composite casts in ROW deparsing Patch Review 4 msgs Nov 20, 08:29
opened Nov 16, 04:26 ·last activity Nov 20, 08:29

The proposer identified a bug where pg_get_viewdef() loses implicit composite type casts when deparsing ROW expressions. This can lead to views failing to restore correctly or silently calling different function overloads after schema changes. The proposer provided a patch that ensures RowExpr printing honors the showimplicit flag for named composites. A reviewer confirmed the bug, tested the patch across multiple PostgreSQL versions, and agreed it merits backpatching. The patch was updated to include more comprehensive test cases, specifically for VARIADIC functions and EXPLAIN output.

3 recent replies
Nov 20, 08:22 The latest reviewer notes that the proposed patch (v2) looks good and its tests are sufficient. The reviewer suggests improving the inline comment to focus on why RECORDOID can be excluded even when showimplicit is true, providing an example of clearer phrasing.
Nov 20, 04:41 The proposer thanked the reviewer for their feedback and testing, then provided an updated patch (v2). This new version incorporates a test case for the VARIADIC scenario, as suggested by the reviewer, and also includes a column-default test. The core implementation remains the same, and the proposer confirmed the approach for back-branching.
Nov 16, 07:09 The latest reply from a reviewer provides a comprehensive positive assessment of the patch. The reviewer confirmed the bug, validated the fix across a wide range of cases, including VARIADIC functions that previously caused dump failures, and agreed that the EXPLAIN changes are a reasonable trade-off. The reviewer also concurred with the proposer that backpatching to versions 15-19 is merited.
archive ↗
22 Per-table resync for logical replication subscriptions Patch Review 37 msgs Nov 19, 05:29
opened Jul 7, 09:41 ·last activity Nov 19, 05:29

The proposer introduced a new ALTER SUBSCRIPTION ... REFRESH TABLE command to allow per-table resynchronization for logical replication subscribers, addressing scenarios like data inconsistency, row filter changes, or repairing a single table without affecting others. The command aims to be subscriber-local and reuse existing tablesync machinery. Throughout numerous patch versions (v1 to v15), the feature was refined through extensive review. Key challenges included preventing deadlocks during concurrent REFRESH operations, ensuring correct handling of replication origins when the publisher is down, managing trigger interactions during TRUNCATE operations, and resolving test race conditions. The proposer implemented solutions such as reordering lock acquisition, adjusting replication origin lifecycle, and refining how triggers are handled. The latest feedback from a new reviewer highlighted that the command currently doesn't verify if the target table is still published, potentially leading to data loss if an unsubscribed table is truncated and cannot be re-copied.

Recent reply
Nov 19, 04:45 A new reviewer identifies a potential problem where REFRESH TABLE does not verify if the named table is still published by the subscription. This can lead to data loss, as the table might be truncated but unable to be re-copied if it's no longer published, with the tablesync worker failing later. The reviewer suggests the command should either remove unsubscribed tables or error out before truncating.
archive ↗
23 pg_restore fails on extracted tar archives with large objects Patch Review 3 msgs Nov 20, 08:29
opened Nov 19, 21:23 ·last activity Nov 20, 08:29

The proposer identified a bug in pg_restore where it fails to restore large objects from extracted tar archives. This issue arises because the tar format stores NNN.dat instead of blobs_NNN.toc in the BLOBS TOC entry, a problem exacerbated by a change in v17. A patch was proposed to fix this by constructing the correct filename instead of relying on the potentially incorrect one in the TOC entry. A reviewer confirmed the bug and the effectiveness of the patch, and also suggested adding a small test. The latest message points out a related problem in _PrepParallelRestore() concerning size estimation for parallel large object restores, suggesting the same filename construction fix might be needed there too.

3 recent replies
Nov 20, 08:04 The latest commenter identifies a related issue in _PrepParallelRestore() where the stat() call fails to get the correct size for BLOBS entries, leaving dataLength at zero. This occurs for the same reason as the main bug (incorrect filename in the TOC entry) and affects parallel scheduler size estimates. The commenter asks if the proposed filename construction logic should also be applied in this function for a complete fix.
Nov 19, 23:11 The latest reply from commenter 1 confirms the bug's reproducibility and verifies that the proposed patch successfully fixes the issue. The commenter tested it with multiple BLOBS batches and parallel execution, finding no regressions. The commenter also suggested considering a small test for `011_restore.pl`, though acknowledged its value might be limited if the tar format is heading towards deprecation.
Nov 19, 21:23 The proposer identified a bug in `pg_restore` that prevents it from restoring extracted tar archives containing large objects due to a mismatch in TOC file naming conventions and a v17 code change. A patch is proposed to construct the correct filename dynamically, and the proposer also suggests deprecating the tar format.
archive ↗
24 pg_upgrade doesn't copy ICU rules on template0 Committed 9 msgs Nov 20, 10:28
opened Oct 28, 12:09 ·last activity Nov 20, 10:28

The thread began with the proposer reporting that pg_upgrade failed to copy ICU rules for template0, an oversight from a previous commit that added ICU rules. A patch author quickly provided a fix, adapting the existing UPDATE mechanism to handle potentially long ICU rules and removing unnecessary version checks in set_locale_and_encoding(). This fix was confirmed by the proposer. A reviewer then raised a broader question about the utility of locale settings (including ICU rules) on template0, as they act as inflexible defaults that cannot be changed after initdb. The reviewer also identified a related bug where CREATE DATABASE with template template0 and LOCALE_PROVIDER='builtin' would fail if template0 had ICU rules, preventing the creation of non-ICU databases. The proposer provided a patch for this CREATE DATABASE issue. Ultimately, both the CREATE DATABASE fix and the original pg_upgrade fix for daticurules were committed, with the reviewer acknowledging that daticurules are now handled consistently with other locale settings despite lingering questions about the overall design.

2 recent replies
Nov 20, 10:24 The latest reply from reviewer 2 confirms the commitment of the original pg_upgrade fix for not copying ICU rules on template0. This fix, based on the v1 patch, ensures daticurules are handled consistently with other locale settings during upgrades. The reviewer reiterates a previous concern about the overall sensibility of template0 holding locale settings but notes that consistency has been achieved.
Nov 19, 04:07 The latest reply from the core developer confirms the commitment of the patch provided by commenter 1. This patch specifically addresses the problem where a cluster initialized with `initdb --icu-rules` would prevent the creation of databases using a different locale provider, as the system incorrectly copied template's ICU rules even for non-ICU databases.
archive ↗
25 [PATCH] Handle SQL NULL results from casts in JSON generation Discussing 2 msgs Nov 20, 06:29
opened Nov 20, 03:36 ·last activity Nov 20, 06:29

The proposer identified an error in PostgreSQL's JSON generation functions where they crash with "function returned NULL" when a custom cast function to `json` or `jsonb` returns a SQL NULL for a non-NULL input. This issue was observed with `json_build_array` and `jsonb_build_array`. The problem stems from `OidFunctionCall1()` treating a NULL result as an error. The initial patch (v1) proposed checking `fcinfo->isnull` and writing a JSON null. In v2, the proposer opted for `to_json()` and `to_jsonb()` to return SQL NULL for such top-level casts, aligning with the behavior of explicit `::json` casts, and is currently seeking feedback on this design choice.

2 recent replies
Nov 20, 06:09 The latest reply from the proposer introduces v2 of the patch. This version proposes that `to_json()` and `to_jsonb()` should return SQL NULL when a cast results in a NULL value at the top level, aligning with the behavior of explicit `::json` casts. The proposer is now soliciting feedback on this specific design choice.
Nov 20, 03:36 The proposer identifies a bug where json_build_array() and jsonb_build_array() fail when a user-defined type cast to JSON explicitly returns SQL NULL, expecting [null] instead. The attached patch calls the cast with FunctionCallInvoke() to detect NULL results and converts them to JSON nulls. A question is posed about the expected top-level SQL NULL behavior for to_json() and to_jsonb().
archive ↗
26 Add a hook for handling logical decoding messages on subscribers. Discussing 49 msgs Nov 20, 05:29
opened Jun 19, 22:33 ·last activity Nov 20, 05:29

The thread proposes adding a hook for logical decoding messages on subscribers to enable extensions to handle DDL, function replication, or trigger custom routines. Early discussions refined the callback signature and transaction ID handling. Later, a reviewer raised a concern about compatibility with older, unpatched publishers when streaming is enabled: messages from aborted subtransactions might not be correctly discarded on the subscriber, as the unpatched publisher sends only the top-level XID. The proposer initially suggested requiring v20+ publishers for the message option. This led to further discussion about nuanced version checks, with one reviewer suggesting different requirements based on whether streaming is enabled or disabled. Concurrently, a new critical issue was identified: non-transactional messages could be repeatedly re-applied on the subscriber due to incorrect LSN advancement. A fix and a test case for this redelivery problem were provided. The proposer has responded to the versioning discussion, opting for a simpler v20+ requirement regardless of streaming, but the redelivery issue remains unaddressed in the latest exchange.

3 recent replies
Nov 20, 04:32 The reviewer agreed with the proposer's decision to simplify version compatibility for the 'messages' option, requiring publishers to be v20 or later irrespective of the streaming setting, finding this approach reasonable.
Nov 19, 23:42 The proposer addresses the ongoing discussion about the publisher version compatibility for requesting messages. They decide to keep the rule simple: require a v20 or later publisher regardless of whether streaming is enabled. This decision is based on avoiding unnecessary complexity and acknowledging that users might not want to forgo the performance benefits of streaming by disabling it for older publishers.
Nov 19, 10:48 The latest reply from commenter 4 discusses the compatibility challenges related to requesting 'messages' from older publishers (v14-v19). They acknowledge that these versions support messages but might have issues when streaming is enabled, specifically concerning aborted subtransaction messages. Commenter 4 weighs options, suggesting either implementing a nuanced version check (requesting messages only from older publishers if streaming is disabled) or potentially accepting the current behavior if the problem is not widespread enough to warrant additional code complexity.
archive ↗
27 Increase NUM_XLOGINSERT_LOCKS Patch Review 9 msgs Nov 19, 15:29
opened Jan 16, 13:52 ·last activity Nov 19, 15:29

This thread discusses increasing the `NUM_XLOGINSERT_LOCKS` value to improve WAL insertion performance. The proposer initially showed measurable gains on a notebook with a synthetic insert/delete workload, suggesting increases up to 256 locks or 64 locks with conditional locking attempts. However, reviewer 1 presented past and current benchmarks on larger servers showing that significantly increasing these locks (e.g., beyond 8) could lead to performance regressions due to increased spinlock contention and cacheline pingpong, especially for certain workloads like logical replication. Despite this, there was some consensus that a smaller increase, like up to 64, might be a 'sweet spot' for smaller installations. Later in the thread, a new proposer (who was also involved in the separate lock-free xlog reservation discussion) submitted a patch (v1) to make `NUM_XLOGINSERT_LOCKS` configurable via a GUC, defaulting to 8 but allowing powers of two from 4 to 128. This patch, submitted independently of a CAS128 reservation patch, showed significant performance gains (up to +40.4%) on high-core count systems when used with 64 locks. The new proposer is seeking review on the GUC setting and its validation.

Recent reply
Nov 19, 15:23 The latest message introduces v1 of a `wal_insert_locks` GUC patch. It proposes making `NUM_XLOGINSERT_LOCKS` configurable, defaulting to 8 but allowing powers of two from 4 to 128, to balance performance gains on large machines with potential regressions. The proposer clarifies its independence from a separate CAS128 patch but provides combined benchmark results showing significant improvements (up to +40.4% with 64 locks) on high-core count systems. Review is requested for the GUC settings and validation.
archive ↗
28 Use local buffer for packed numerics Discussing 5 msgs Nov 20, 08:29
opened Nov 17, 09:58 ·last activity Nov 20, 08:29

The proposer introduced a patchset to optimize numeric operations by using local buffers for unpacking MinimalTuple numerics, preventing repeated palloc/pfree calls. Initial benchmarks indicated 15-35% speedup, particularly for aggregates and sorting. A reviewer pointed out a more aggressive optimization, working directly with packed numeric representations, and shared their own patch for comparison. The discussion then expanded to cover the proposer's broader, three-part numeric optimization project, including direct comparison for numeric_eq()/numeric_ne() which showed even greater speedups. The latest messages involve sharing benchmark results and alternative patch approaches from the reviewer, highlighting mixed performance outcomes depending on architecture and implementation details, with a focus on centralizing case distinctions.

3 recent replies
Nov 20, 08:14 The latest commenter shares benchmark results for their proposed "not unpacking" optimization on an AMD Ryzen 7 9700X. The results show significant speedups but also regressions, which the commenter attributes to repeated case distinctions in existing macros. A new patch demonstrating an alternative approach to centralize this parsing logic within init_var_from_num() is also attached, aiming for cleaner and more consistent code.
Nov 19, 17:33 A commenter requests the proposer to execute their changes against a benchmark script, specifically asking for separate results for the scratch buffer optimization and then combined with avoiding unpacking entirely. The commenter also asks about the testing architecture used by the proposer. They mention that their own patch for avoiding unpacking showed good performance on x86 but regressions on M5, indicating platform-specific challenges. The commenter is actively working to centralize the case distinction logic for better cross-platform robustness.
Nov 18, 10:01 The proposer thanked commenter 1 for the review and provided more context on their overall numeric optimization project, which includes three parts: local buffers (this thread), direct comparison from packed bytes (draft attached), and a fast 128-bit accumulator (not yet ready). The proposer believes the direct comparison part shows even greater speedups (30-60%) and views commenter 1's suggestion of direct packed representation work as complementary rather than an alternative. The proposer requested commenter 1's patch to compare approaches.
archive ↗
29 CI slowdown due to PG_TEST_INITDB_EXTRA_OPTS Committed 18 msgs Nov 20, 08:29
opened Jul 18, 16:38 ·last activity Nov 20, 08:29

The thread identifies that the use of PG_TEST_INITDB_EXTRA_OPTS in CI jobs significantly slows down test execution by disabling the initdb template mechanism, leading to a 3x increase in CPU time. Initial proposals included recording and matching template options or introducing a new environment variable PG_TEST_SERVER_POSTGRES_OPTS to directly set GUCs in postgresql.conf. The latter approach was chosen and a patch was committed, leading to substantial performance improvements (20-30% reduction in CI runtimes). Discussions also occurred regarding further optimizations, such as splitting long-running ASAN and MinGW tests across multiple runners. Following the commitment of the primary patch, there's an ongoing discussion about backpatching this improvement to older supported PostgreSQL versions, with concerns raised about introducing new user-visible options to stable branches.

3 recent replies
Nov 20, 07:49 The latest commenter explains their hesitation to backpatch the recently committed PG_TEST_SERVER_POSTGRES_OPTS feature to older stable branches. The reason given is that it introduces a new user-visible option to pg_regress, which they consider a new feature rather than solely a CI-specific change, making backpatching less straightforward.
Nov 19, 07:57 The latest reply from commenter 3 points out a potential issue with backpatching the recently committed PG_TEST_SERVER_POSTGRES_OPTS change. The commenter notes that PG_TEST_INITDB_EXTRA_OPTS, which is related to the context of the change, was introduced in PG_17 and has not been backpatched to older versions (e.g., PG_15, where CI was introduced), suggesting this might complicate the backpatching effort.
Nov 18, 20:07 A committer announced that the patch implementing `PG_TEST_SERVER_POSTGRES_OPTS` had been committed. Another participant then requested that this fix be back-patched, in preparation for back-patching CI changes, to minimize differences between branches, suggesting there are no arguments against it.
archive ↗
30 [PATCH] psql: Complete GRANTED BY in GRANT and REVOKE Proposed 1 msgs Nov 20, 09:28
opened Nov 20, 08:57 ·last activity Nov 20, 09:28

The proposer identified two completion issues within psql for `GRANT` and `REVOKE` commands. Firstly, the `GRANTED BY` clause and grantor names were not being completed. Secondly, when typing `GRANT SELECT ON t TO r WITH GRANT OPTION` and pressing Tab after a space, psql incorrectly inserted `TO`, invalidating the statement. The proposed patch aims to resolve these by adjusting the completion logic to prioritize matching `WITH GRANT OPTION` and by adding proper completion for the `GRANTED BY` clause and grantor names in both `GRANT` and `REVOKE` commands.

Recent reply
Nov 20, 08:57 The proposer submitted a patch to fix psql's tab completion for `GRANT` and `REVOKE`. The issues included missing completion for `GRANTED BY` and grantor names, and incorrect insertion of `TO` after `WITH GRANT OPTION`, leading to invalid statements. The patch aims to rectify these behaviors, improving user experience.
archive ↗
31 pg_dump: remove restore.sql from tar format Patch Review 1 msgs Nov 20, 09:28
opened Nov 20, 07:39 ·last activity Nov 20, 09:28

The proposer suggested removing the undocumented `restore.sql` script from `pg_dump`'s tar format. This script has historically caused maintenance burden, requiring numerous fixes and necessitating a costly second pass over the Table of Contents (TOC) with tar-specific hacks that re-run attribute statistics queries. The proposer argued that `pg_restore -f` offers a supported and more efficient alternative for generating a restore script. Commenter 1 strongly supported the removal, agreeing that the overhead associated with an undocumented script, particularly the second TOC pass and tar-specific machinery, is not justified given the existence of a superior, supported alternative.

Recent reply
Nov 20, 07:39 Commenter 1 provided positive feedback, agreeing with the proposer's suggestion to remove the `restore.sql` script from `pg_dump`'s tar format. The commenter deemed the costs of maintaining the undocumented script, such as the second TOC pass and tar-specific logic, to be unwarranted given that `pg_restore -f` provides a better, supported alternative.
archive ↗
32 Initial table sync copies the whole table when it is no longer published Discussing 19 msgs Nov 20, 05:29
opened Nov 16, 09:44 ·last activity Nov 20, 05:29

The proposer identified a bug where, during initial logical replication table synchronization, if a publication is dropped or a table is removed from it before sync begins, the subscriber incorrectly copies the entire table, ignoring column filters and row specifications. The proposed fix introduces an error in such cases, preventing erroneous data copy and ensuring the table is not silently marked as synchronized. Extensive discussion ensued regarding whether to raise an ERROR or a WARNING, and how users could recover from such errors, especially with disable_on_error enabled. A workaround for recovery was detailed and subsequently documented, with refinement suggestions for the documentation itself. Test cases were also fine-tuned. The core patch appears to be accepted, but the latest message from the proposer questions the necessity of backpatching, citing precedents where similar behavioral changes were not backpatched without field reports, suggesting further discussion or waiting for feedback.

3 recent replies
Nov 20, 03:32 The proposer agreed to provide back-branch patches if the approach is approved, but also raised a question about the necessity of backpatching this specific issue. They referenced a similar past commit that was not backpatched due to it being a behavior change and the absence of field reports, suggesting a cautious approach and waiting for further community feedback.
Nov 19, 10:33 The latest reply from reviewer 2 confirms their agreement to retain the `REFRESH PUBLICATION` block within the TAP test. This decision aims to thoroughly test the recovery mechanism in scenarios where an initial table synchronization fails, as recommended by the patch's error hint and documentation. Reviewer 2 also requested back-branch patches, indicating the perceived importance of this fix for multiple PostgreSQL versions.
Nov 19, 09:26 The latest reply from a developer confirms agreement with the suggestions for test case removal. The developer states that the inserts after the error and the REFRESH PUBLICATION block can be removed from the test script, as they do not effectively test the fix, and the wait_for_log checks already confirm the sync failure.
archive ↗
33 Avoid copying MinimalTuple twice in ExecHashSkewTableInsert() Patch Review 2 msgs Nov 20, 08:29
opened Nov 18, 14:56 ·last activity Nov 20, 08:29

The proposer identified an inefficiency in ExecHashSkewTableInsert() where MinimalTuple was copied twice when the input slot was not a MinimalTupleTableSlot. A patch was proposed to use ExecCopySlotMinimalTupleExtra() to avoid the redundant copy, aiming for minor performance improvements and code simplification.

2 recent replies
Nov 20, 08:21 The reviewer confirms that the patch avoids a redundant allocation, memcpy(), and pfree() when slot->tts_ops->get_minimal_tuple is NULL. The reviewer concludes that the patch slightly improves performance and simplifies the code, making it a good improvement, and reports that tests passed.
Nov 18, 14:56 The proposer identifies an inefficiency in `ExecHashSkewTableInsert()` where `MinimalTuple` is copied twice if the input slot is not a `MinimalTupleTableSlot`. A patch is attached to resolve this by using `ExecCopySlotMinimalTupleExtra()` to avoid the redundant copy.
archive ↗
34 Proposal: Conflict log history table for Logical Replication Patch Review 48 msgs Nov 20, 04:28
opened Oct 26, 09:14 ·last activity Nov 20, 04:28

This lengthy thread discusses the proposal and implementation of a conflict log history table for Logical Replication. Key issues include handling cases where replica identity values render to more than 1GB during conflict recording, which has been addressed with documentation. A significant part of the discussion revolves around how the apply worker interacts with prepared transactions, particularly concerning locks on the Conflict Log Table (CLT). Commenters debate whether to error out the apply worker due to potential false positives, document the hazards, or implement more precise lock-acquisition checks, with no definitive consensus yet. Another area of focus is pg_dump and pg_upgrade behavior for the CLT, including dumping definitions, privileges, comments, and security labels, and ensuring correct testing. The patches are undergoing extensive review, with multiple versions (up to v96 mentioned) addressing various code, documentation, and testing comments.

3 recent replies
Nov 20, 03:39 The latest reply from reviewer 1 suggests that the catalogs.sgml issue, regarding missing new pg_subscription columns (subconflictlogrelid and subconflictlogdest), should be grouped with other describe-subscription changes in patch 0003 instead of being a separate fix. This indicates a focus on logical patch organization as part of the ongoing detailed review.
Nov 19, 14:21 The main implementer addresses the latest review comments on patch v95. They suggest fixing the catalogs.sgml omission in a separate patch. Other comments, such as refining column documentation and adding code explanations, are incorporated into the v96 patch, which specifically updates the 0001 patch.
Nov 19, 10:06 The latest reply provides several detailed review comments on the current patch version. These include identifying missing column documentation in `catalogs.sgml`, suggesting clearer wording for function comments and error messages related to the conflict log table, pointing out a stray blank line in `executor.h`, and correcting two typos within code comments. These comments aim to further refine the patch for accuracy and clarity.
archive ↗
35 Injection point: fix race condition between attach and cleanup Rejected 11 msgs Nov 20, 02:29
opened Nov 11, 12:26 ·last activity Nov 20, 02:29

This thread discusses a race condition in PostgreSQL's injection point mechanism, specifically during the cleanup of local injection points in multi-backend test scenarios. The proposer observed that an exiting backend's cleanup routine might inadvertently detach a newly attached injection point if another backend reused the same name, leading to incorrect test behavior. A patch was proposed to uniquely identify injection points using a slot ID and generation to prevent this. However, core developers strongly pushed back against complicating the server-side injection point functionality, arguing that test facilities should remain simple. They suggested that such issues should either be handled within the test code itself, by using distinct names for injection points, or by improving the test infrastructure (e.g., ensuring backends fully exit before reuse). The consensus was to reject the proposed core changes due to added complexity and to address the problem at the testing layer, either by modifying test scripts or enhancing test utility functions.

3 recent replies
Nov 20, 02:12 The latest replies reinforce the consensus that the proposed core changes to fix the injection point race condition are not desirable. Multiple reviewers agreed that complicating the core infrastructure for a testing-specific issue is unwarranted. They reiterated that the problem should be addressed at the test level, either by ensuring proper backend exit and cleanup timing or by requiring distinct names for injection points within test scripts.
Nov 20, 00:41 The latest reply (reviewer 2) suggests that a default guarantee for BackgroundPsql.quit to wait for the corresponding backend to exit would be beneficial for the entire testing infrastructure. This would address the race condition by ensuring cleanup callbacks are completed before resources are reused, which is also relevant for other tests that depend on the timing of shmem exit callbacks.
Nov 19, 00:33 The latest reply from the proposer agrees that the proposed patch is too complex. It discusses a simpler alternative: modifying test utilities like `BackgroundPsql.quit` to optionally wait for the corresponding backend to exit. This would provide a reusable way to address the race condition in tests like `010_index_concurrently_upsert` and `051_effective_wal_level.pl` without adding complexity to the core injection point mechanism. The proposer also suggests using distinct names or explicit detaching to avoid the issue in isolation tests.
archive ↗
36 [PATCH]Fix planner's strictness check for JsonExpr Patch Review 17 msgs Nov 20, 02:29
opened Aug 24, 10:13 ·last activity Nov 20, 02:29

The thread addresses a bug in PostgreSQL's planner where `JsonExpr` nodes were incorrectly treated as strict, leading to incorrect `NULL` handling, particularly in `LEFT JOIN` scenarios with `PASSING` arguments. The initial proposal aimed to make `JsonExpr` non-strict by default. Through several iterations and detailed reviews, the strictness rule for `JsonExpr` was refined. Key discussions revolved around correctly handling `RETURNING` clauses with null-rejecting domains, and the interaction with other planner optimizations like expression inlining. The solution evolved to define `JsonExpr` as non-strict if it returns any domain, or if a variable, placeholder, or parameter appears in its `PASSING` values or `ON EMPTY / ON ERROR` expressions. This allows more precise strictness checks, enabling inlining for more `JSON` expressions while preserving correctness. The patch also considers its integration with related ongoing work, such as a separate fix for `FULL JOIN` strictness.

3 recent replies
Nov 20, 01:55 The proposer submitted v6, incorporating feedback to remove dependency on the `use_json_coercion` flag and instead checking only the `RETURNING` type. The strictness rule was further refined to treat a `JsonExpr` as non-strict if it returns any domain, or if a `Var`, `PlaceHolderVar`, or `Param` appears in its `PASSING` values or `ON EMPTY / ON ERROR` expressions. This version passes all tests, enables more inlining, and is rebased and co-authored.
Nov 19, 14:56 The latest message from reviewer 1 acknowledges the ongoing discussion around the `JsonExpr` strictness fix. It notes a dependency on the `use_json_coercion` flag, which reviewer 1 plans to remove in a separate patch, causing a conflict. Reviewer 1 also mentions a related `FULL JOIN` fix which has been pushed independently, emphasizing the need for a consistent solution for `JsonExpr` across relevant code paths.
Nov 19, 13:40 The reviewer notes that the proposed patch (v5) conflicts with an upcoming patch that removes the use_json_coercion flag, on which v5 relies. They also point out that the use_json_coercion flag's setting in the parser implicitly depends on DomainHasConstraints(), suggesting a potential discrepancy with v5's stated independence from domain constraints.
archive ↗
37 Use an index scan in RemoveSubscriptionRel() Committed 6 msgs Nov 20, 00:29
opened Nov 17, 05:32 ·last activity Nov 20, 00:29

The thread discusses a performance improvement for RemoveSubscriptionRel() on a subscriber, proposing to replace an inefficient heap scan with an index scan when looking up by relid. The proposer demonstrated significant performance gains, reducing operation time from 2.8s to 0.23s in a synthetic test with 100,000 rows. Initial review questioned the necessity of reordering scan keys in the patch. However, a commenter proved this reordering is essential by demonstrating a failure without it and explaining the underlying _bt_preprocess_keys() check. The proposer confirmed that existing test suites also caught this issue. The patch was updated and subsequently committed.

3 recent replies
Nov 20, 00:25 The commenter expresses self-reproach for not checking existing regression tests earlier. They state that the issue concerning the necessity of scan key reordering in the patch was already covered by the existing test suite, making their custom TAP script redundant.
Nov 19, 19:17 The proposer confirmed that the updated patch from commenter 1 addressed the identified issues and looked correct. The proposer also added that the specific test case provided by reviewer 2 to demonstrate the necessity of scan key reordering was already covered by existing PostgreSQL test suites, which would fail with the same "btree index keys must be ordered by attribute" error on unpatched systems.
Nov 19, 07:35 A developer clarified a previous reviewer's comment regarding the necessity of reordering scan keys in the patch. The developer confirmed that the reordering is required, providing a test script that fails without it and referencing an internal check (_bt_preprocess_keys()) that enforces key ordering by attribute number during btree index scans. The developer agreed, however, that the accompanying comment about reordering is unnecessary.
archive ↗
38 [PATCH v1] Fix initial sync of zero-column published tables Patch Review 13 msgs Nov 20, 06:29
opened Nov 1, 04:19 ·last activity Nov 20, 06:29

The proposer identified a bug where initial synchronization of a zero-column published table mishandles subscriber-only columns, either populating them with empty strings instead of defaults (text mode) or failing with a field-count mismatch (binary mode). This occurs because `COPY` incorrectly interprets an empty column list as an unspecified list. The proposed solution involves introducing `BeginCopyFromExtended()`, which accepts a flag to explicitly indicate an empty column list, ensuring subscriber defaults are correctly applied. The patch progressed through several versions. Initial reviews by commenter 1 and commenter 2 provided feedback on test simplification and code comments, leading to v2 and v3. Reviewer 3 confirmed the fix on master and verified it corrected various edge cases. Further discussions with commenter 2 led to an improved commit message (v4) and requests for backporting patches to affected PostgreSQL branches. The proposer then provided v5, with separate patches for master and REL_15_STABLE, addressing all previous comments and confirming extensive testing across multiple branches.

3 recent replies
Nov 20, 06:20 The latest reply from the proposer provides v5 patches for both the master branch and the REL_15_STABLE branch. This update incorporates previous reviewer suggestions regarding cosmetic changes, refined NIL comments in the `COPY` entry points, and addresses the explicit requirement for backporting to affected branches. The proposer also confirms that the patch has undergone successful testing across multiple PostgreSQL versions.
Nov 19, 04:35 A reviewer acknowledges the updated patch (v4) and its improvements. However, they note that the patch does not apply cleanly to the PG15 branch, which is also affected by the issue. They request a separate patch specifically for PG15 to facilitate back-patching.
Nov 19, 00:07 The latest reply, from reviewer 1, offered minor cosmetic suggestions for the v4 patch. These included removing redundant version change notes from the commit message, rephrasing internal comments in `copyfrom.c` for better clarity on how a `NIL attnamelist` is interpreted, and simplifying a comment within the `008_diff_schema.pl` TAP test file.
archive ↗
39 Switching XLog source from archive to streaming when primary available Discussing 9 msgs Nov 19, 11:29
opened Jan 2, 18:12 ·last activity Nov 19, 11:29

This thread discusses a proposed feature for PostgreSQL standbys to automatically switch from archive recovery to streaming replication when the primary becomes available. Initially, the discussion focused on the performance differences between archive and streaming replication. However, the proposer later shifted the focus to a correctness/availability problem: a standby stuck in archive recovery can prevent its physical replication slot on the primary from advancing, leading to unbounded WAL retention on the primary and potential disk exhaustion, as well as stalling logical replication if slot synchronization is enabled. A patch (v25) was introduced that adds a timeout-based parameter, 'streaming_replication_retry_interval', allowing the standby to attempt switching to streaming after consuming local WAL, falling back to archive if streaming fails. A reproducer for the issue was provided. The latest discussion explores an alternative approach: having the walreceiver on the standby send feedback to the primary even while processing WAL from the archive, to advance the replication slot without necessarily switching to streaming.

Recent reply
Nov 19, 10:59 The latest reply from commenter 3 suggests an alternative or complementary approach where the walreceiver on the standby could operate in a feedback-only mode. This would allow the standby to report its WAL replay progress to the primary while still reading from the archive, thereby advancing the replication slot and preventing WAL retention issues without forcing an immediate switch to streaming. This approach could also potentially work in conjunction with the proposed timeout-based switching mechanism.
archive ↗
40 [PATCH] Clamp auto-tuned transaction_buffers to at least 8 SLRU banks Proposed 1 msgs Nov 20, 06:29
opened Nov 20, 06:08 ·last activity Nov 20, 06:29

The proposer identified a performance bottleneck in PostgreSQL related to the auto-tuning of `transaction_buffers`. When `transaction_buffers` is set to 0, the `CLOGShmemBuffers()` function can allocate a very small buffer pool, leading to only two SLRU banks for common `shared_buffers` configurations. This small pool causes frequent eviction and re-reading of CLOG pages, leading to high contention on the `LWLock:XactSLRU` lock and reduced transaction throughput under high concurrency. Benchmarking on a 128-core machine with `pgbench` showed significant performance degradation (up to 8.1% lower TPS) and millions of CLOG page re-reads. The proposed patch clamps the auto-tuned `transaction_buffers` to a minimum of 8 SLRU banks (128 slots, 1MB), which was found to eliminate pathological re-read behavior and restore full TPS with minimal memory overhead.

Recent reply
Nov 20, 06:08 The latest reply is the initial proposal, presenting a patch to clamp the auto-tuned `transaction_buffers` to a minimum of 8 SLRU banks (128 slots, 1MB). This change aims to mitigate severe performance degradation observed with smaller auto-tuned buffer pools under high concurrency, which currently cause excessive CLOG page re-reads and LWLock contention.
archive ↗
41 WAL_LOG CREATE DATABASE strategy broken for non-standard page layouts Committed 28 msgs Nov 20, 00:29
opened Dec 10, 06:25 ·last activity Nov 20, 00:29

The thread, which has been ongoing for a long time, addresses a bug in CREATE DATABASE's WAL logging strategy that mishandles non-standard page layouts, leading to issues with Visibility Map (VM) pages. The proposed solution is a hybrid approach, using page_std for heap tables and btree indexes, and zeroing out unused space. Recent discussions focused on refining the TAP tests to ensure original coverage is maintained and to make them less fragile by increasing shared_buffers. A related issue was discovered where wal_consistency_checking did not properly validate VM pages, which is being considered for a separate fix. The main fix has been committed and backpatched. Further discussion is on an additional optimization for RelationCopyStorage() for the master branch.

3 recent replies
Nov 20, 00:01 Commenter 1 responds to an earlier suggestion about increasing shared_buffers for a specific test scenario. They suggest that instead of relying on custom buildfarm configurations for this level of tuning, it would be more effective to conditionally bump shared_buffers directly within the test framework when wal_consistency_checking is enabled, as this mode already consumes significant resources.
Nov 19, 21:42 The proposer confirms that the main fix for the `WAL_LOG CREATE DATABASE` strategy, addressing issues with non-standard page layouts, has been successfully committed and backpatched. The remaining attached patch is specifically for the master branch and introduces an optimization to `RelationCopyStorage()`, including a new test, signifying the ongoing refinement of the system following the primary fix.
Nov 18, 15:41 A reviewer suggested adding comments to the CREATE DATABASE command in the test to provide developers with a clear hint. Additionally, the reviewer proposed an optional check after the command to verify that pg_class pages remain in the buffer cache, by comparing block fetched and hit counts, provided autovacuum is disabled.
archive ↗
42 Parallel vacuum: check XID wraparound failsafe between indexes Patch Review 12 msgs Nov 20, 03:28
opened Oct 16, 16:04 ·last activity Nov 20, 03:28

The thread discusses a proposed improvement for parallel vacuum in PostgreSQL 20 to check the XID wraparound failsafe more frequently. Currently, this check occurs only after all parallel workers have vacuumed their assigned indexes, potentially delaying the advancement of `relfrozenxid` and `relminmxid` and increasing the risk of XID wraparound. The proposer suggests a mechanism where both the leader and parallel workers check the failsafe after each index and set a shared flag if triggered. This flag would then cause cost-based delays to be ignored and remaining indexes to be skipped, ensuring faster completion and XID advancement. Early discussion validated the approach and suggested minor code improvements. A reviewer later identified a potential issue where dynamically changed `vacuum_failsafe_age` GUC values might not be honored by workers. The proposer addressed this in v4 by ensuring the failsafe age is passed from the leader to workers so they can react to such changes.

2 recent replies
Nov 20, 02:56 The latest patch, v4, addresses a concern raised by a reviewer regarding the `vacuum_failsafe_age` GUC. This version ensures that the failsafe age is passed from the leader to the parallel workers. This allows both the leader and workers to honor any changes to the failsafe age, thereby preventing situations where a dynamically lowered failsafe age might be missed by workers during long vacuum operations.
Nov 18, 17:29 The proposer responded to a reviewer's feedback regarding how `vacuum_failsafe_age` changes are handled in parallel vacuum. They clarified that the issue is not with an extra check in the leader, but with workers not honoring GUC changes since values are serialized at context setup. The proposer plans a better fix by propagating the failsafe age from the leader to workers and will also enhance the TAP test to check for specific warning messages, ensuring the failsafe is correctly reported.
archive ↗
43 Index drops during phase 1 of REINDEX CONCURRENTLY Committed 6 msgs Nov 20, 05:29
opened Nov 14, 00:31 ·last activity Nov 20, 05:29

The proposer identified a problem in REINDEX CONCURRENTLY where indexes could be dropped during the first phase of re-indexing, leading to errors in the second phase. The initial patch aimed to address this, but a reviewer found a scenario where the fix was insufficient due to RelationGetIndexList() returning indexes in OID order and DROP INDEX being possible within the same transaction. This could cause an index to be dropped by a function call while another index was being processed, leading to subsequent failures. After discussion and a proposed alternative solution by a commenter, the proposer decided to keep the Relation references open from the initial index gathering loop, simplifying the approach. The proposer then committed this revised fix, including backpatching to v15.

3 recent replies
Nov 20, 04:47 The proposer reverted the test ordering to its original intent, stating that a commenter's patch was overcomplicating the solution. The proposer committed a revised approach where index relations are kept open from the initial gathering loop and thanked all participants for their reviews.
Nov 16, 04:04 The latest reply from commenter 2 presents a refined patch based on previous discussions. This patch aims to resolve an issue where an index could be dropped during REINDEX CONCURRENTLY if its OID is processed later, by explicitly opening all original target indexes before the phase 1 creation loop begins. The patch also adjusts the test case to reflect the index creation order that exposes this problem, and it passes with the proposed change.
Nov 15, 06:18 The proposer agreed with reviewer 2's suggestion to open all old indexes before the reindexing loop to prevent index_open() failures when an index is dropped out of order. The proposer invited reviewer 2 to prepare a follow-up patch implementing this idea, noting that they would be unavailable the following week.
archive ↗
44 Autovacuum freeze scores wrap around for just-vacuumed tables Committed 5 msgs Nov 19, 15:29
opened Nov 11, 14:47 ·last activity Nov 19, 15:29

This thread addresses a bug in `relation_needs_vacanalyze()` where autovacuum freeze scores could incorrectly wrap around to a very large value for recently vacuumed tables. The proposer found that if `relfrozenxid` or `relminmxid` advanced due to a concurrent `VACUUM` or table rewrite between reading `recentXid` and `pg_class` scan, the subtraction would overflow. This could cause just-vacuumed tables to be assigned an unduly high priority in the autovacuum list. The proposer provided a patch to treat the age as zero in such overflow cases and included a TAP test for demonstration. Commenter 1 confirmed the issue and generally approved the patch, suggesting back-patching. Reviewer 1 agreed the fix was valuable, though not a release blocker, and committed the patch.

Recent reply
Nov 19, 15:21 The committer announced that the patch addressing the autovacuum freeze score wrap-around issue has been committed. This resolves the problem where recently vacuumed tables could be incorrectly prioritized due to an overflow in age calculation.
archive ↗
45 Fix resource leaks in pg_receivewal on LZ4 close failures Discussing 10 msgs Nov 19, 13:28
opened Nov 11, 09:48 ·last activity Nov 19, 13:28

The proposer initiated a discussion regarding file descriptor and memory leaks in `pg_receivewal` when LZ4 compression finalization or the final write operations fail. A patch was proposed to address these resource leaks. A commenter raised a question about the patch's behavior in `dir_close()`, where it appeared to bypass cleanup operations for a successful `close()`. The proposer clarified this was intentional, aiming to preserve existing error handling behavior while still releasing resources. Another commenter supported the patch, highlighting the importance of fixing resource leaks in a service that might run in a loop. Further discussion ensued about a potential temporary file leak (unlink) that the current patch might not fully address, leading to a suggestion for a separate follow-up patch to handle this broader cleanup scenario. The proposer agreed, indicating that a separate patch would be more appropriate for that specific issue.

3 recent replies
Nov 19, 12:44 The commenter confirmed they would proceed with preparing a follow-up patch. This patch aims to address a broader temporary file leak scenario, encompassing `close()` and `gzclose()` failures, which was identified as a separate concern from the initial LZ4 resource leak fix.
Nov 19, 07:42 The latest reply from commenter 1 acknowledges the proposer's suggestion to address the temporary file leak in a separate follow-up patch. The commenter then asks the proposer if they intend to prepare this follow-up patch themselves.
Nov 19, 07:19 The latest reply from the proposer addresses a concern about a potential temporary file leak. They acknowledge the issue but suggest it's a broader cleanup behavior problem, extending beyond just LZ4 errors, and propose to tackle it in a separate patch to keep the current fix focused on LZ4 resource leaks.
archive ↗
46 A latent pointer increment bug in pgtypes_defmt_scan() Committed 4 msgs Nov 20, 01:28
opened Nov 19, 07:22 ·last activity Nov 20, 01:28

The proposer discovered a latent bug in pgtypes_defmt_scan() where a while loop intended to skip leading spaces was incorrectly advancing pstr (a char**) instead of *pstr (the char* pointing to the string). This bug could potentially cause a crash but was effectively dead code because the caller already skipped leading spaces. The proposer provided a one-line fix. Another commenter confirmed the fix was correct but suggested just removing the loop and adding an assertion. The committer agreed the code was wrong and committed the one-line fix, noting that the function could use more cleanup due to redundant space-skipping logic. The proposer acknowledged the redundancy but preferred keeping the function self-contained for now.

3 recent replies
Nov 20, 00:48 The latest reply (the proposer) acknowledges the committer's observation about redundant code but prefers to keep the function self-contained with the committed one-line fix rather than embarking on further cleanup that would involve changes to the caller.
Nov 19, 10:38 In the final message of the thread, the committer acknowledged the identified pointer increment bug and its incorrect behavior. They confirmed that the proposed one-line fix was correct. Despite recognizing that the code could benefit from a broader cleanup, the committer decided to apply the immediate one-line fix to address the specific issue.
Nov 19, 08:33 A reviewer confirmed that the proposed fix is correct and describes the issue as a 'typical pointer mistake.' The reviewer also suggested an alternative approach: removing the loop entirely and adding an assertion to ensure the input string has no leading spaces, since the current caller already performs this check.
archive ↗
47 Block various aggregate component functions from being called directly Rejected 4 msgs Nov 19, 22:28
opened Nov 19, 07:20 ·last activity Nov 19, 22:28

This thread discusses a proposal to block various aggregate component functions from being called directly by adding a `prointernal` column to `pg_proc`. The proposer observed an increase in bug reports from LLMs and fuzz testers for these internal functions, which are not designed for direct user interaction and can exhibit unexpected behavior in edge cases. The proposed solution aimed to prevent such misuse while still allowing their use within `CREATE AGGREGATE` or `CREATE OPERATOR`. Commenter 1 supported the trade-off. Reviewer 1 suggested using a `prokind` value instead of a new column and advocated for allowing extensions to use this feature. However, commenter 2 demonstrated that the proposed method would not prevent the core issue, as these functions could still be misused by creating a custom aggregate that wraps them, rendering the `prointernal` flag ineffective for stopping such bug reports. This fundamental flaw led to the rejection of the proposed approach.

3 recent replies
Nov 19, 22:04 The latest reply from commenter 2 strongly questions the effectiveness of the proposed solution. Commenter 2 demonstrates that even with the patch, the problematic functions can still be called indirectly via `CREATE AGGREGATE`, allowing LLMs or fuzzers to find and report the same 'bugs'. This suggests the patch would not achieve its goal of preventing these types of reports and might even draw more attention to the area.
Nov 19, 15:05 The latest message provides feedback on the proposal to block direct calls to internal aggregate functions. Reviewer 1 supports the core idea, suggesting a new `prokind` value as an alternative to a new `pg_proc` column. The reviewer also advocates for allowing extensions to mark their own functions as internal, against the proposer's initial suggestion to limit this to built-in functions.
Nov 19, 08:16 The latest reply from commenter 1 expresses agreement with the proposer's sentiment regarding the "bug reports" and considers the proposed patch a reasonable trade-off given limited development resources. The commenter views this approach as a good way to manage unintended uses of internal functions.
archive ↗
48 auto_explain crashes on EXPLAIN (BUFFERS) of an index scan (19, master) Committed 3 msgs Nov 20, 03:28
opened Nov 14, 03:44 ·last activity Nov 20, 03:28

A crash in `auto_explain` was reported when using `EXPLAIN (BUFFERS)` without `ANALYZE` for index scans, while `log_analyze` is enabled. This issue affected PostgreSQL versions 19 and master, tracing back to a change where index scan nodes stopped allocating `IndexScanInstrumentation` under `EXEC_FLAG_EXPLAIN_ONLY`. The `auto_explain` extension, however, still attempted to print these plans as if they had instrumentation, leading to a NULL pointer dereference. The proposer provided a patch that treats missing instrumentation as zero counts within `explain.c`, logging `(never executed)` and `Index Searches: 0` as expected, and included new TAP test cases. The patch was reviewed, confirmed, and subsequently committed to resolve the crash.

3 recent replies
Nov 19, 22:54 Reviewer 1 confirmed the validity of the patch provided by the proposer. Following the review, the reviewer committed the fix to the codebase. This action resolves the `auto_explain` crash that occurred during `EXPLAIN (BUFFERS)` without `ANALYZE` for index scans when `log_analyze` was enabled.
Nov 14, 16:27 A committer acknowledged the bug report and the proposed patch. They stated that they would investigate the issue in the following week, after the PostgreSQL 19.0 release, as its branch was currently frozen and therefore no changes could be made immediately.
Nov 14, 03:44 The proposer identified a crash in `auto_explain` when using `EXPLAIN (BUFFERS)` on index scans in PostgreSQL 19 and master. This occurs because `IndexScanInstrumentation` is not allocated under `EXEC_FLAG_EXPLAIN_ONLY`, causing `show_indexscan_info()` to dereference a NULL pointer. The proposer provided a patch for `explain.c` to treat missing instrumentation as zero counts, restoring prior behavior, and included new TAP tests to validate the fix.
archive ↗
49 [PATCH v2] Fix SP-GiST index-only scans that return only INCLUDE columns Committed 2 msgs Nov 19, 21:28
opened Nov 11, 11:17 ·last activity Nov 19, 21:28

The proposer identified a critical bug in SP-GiST index-only scans when querying only `INCLUDE` columns, especially when the opclass cannot reconstruct the key (`canReturnData = false`). In such scenarios, `leaf_consistent` would still attempt to return the key, which `heap_form_tuple` would misinterpret, leading to data corruption or crashes. The proposer provided a patch (v2) that ensures the key is only requested from `leaf_consistent` when `canReturnData` is true, otherwise returning NULL, and prevents the key from being copied into the ordered-scan queue. A similar fix had been incorporated into the new `spggetbatch` path, but this patch was specifically adapted for the existing `spggettuple` path for backpatching. The bug has existed since `INCLUDE` support was added to SP-GiST in v14, and the proposer recommended backpatching through v15. The patch was tested and confirmed to resolve the issue.

Recent reply
Nov 19, 21:10 The latest reply confirms that a slightly tweaked version of the proposer's patch has been pushed. This indicates that the proposed fix, addressing the bug in SP-GiST index-only scans concerning `INCLUDE` columns when the key cannot be reconstructed, has been accepted and committed.
archive ↗
50 Keep TIDBitmap pagetable memory across bitmap scans Rejected 7 msgs Nov 20, 02:29
opened Nov 9, 16:41 ·last activity Nov 20, 02:29

The thread discusses performance bottlenecks in `TIDBitmap` due to repeated memory allocations and deallocations, leading to significant page faults and kernel contention, especially under high concurrency. The proposer identified that the `TIDBitmap` pagetable, a simple hash table, frequently grows from a small size, causing numerous memory allocations and subsequent trims, leading to repeated page faults. Initial measurements showed substantial system time and spinlock contention. The proposer explored various solutions, including presizing the table and tuning `malloc` settings, but found these inadequate. A prototype patch was proposed to reuse allocated memory for up to two large bitmaps per backend. Further investigation ruled out cgroups as the primary cause of contention, instead pointing to per-NUMA-node page queue locks on page faults and releases. Testing with huge pages confirmed they could eliminate page faults but did not address other scaling issues related to the `TIDBitmap`'s doubling growth strategy. The proposer concluded that the ultimate solution likely involves a more fundamental data structure change (radix tree) combined with process-level fault fixes, and offered to contribute the test module to the radix tree effort. The discussion shifted towards foundational changes, de-prioritizing the current patch.

3 recent replies
Nov 20, 02:24 The latest reply from a core developer indicates that the immediate proposal of reusing `TIDBitmap` memory is not being pursued directly. Instead, the developer plans to return to a prerequisite patch for parallel bitmap index scans, which is a necessary step before working on a more fundamental change to the `TIDBitmap` data structure (a radix tree), as suggested by the proposer.
Nov 17, 20:11 The proposer confirmed that huge pages can eliminate page faults for TIDBitmap allocations and related operations like hash joins and sorts. However, they noted that huge pages do not remove the remaining overhead associated with the TIDBitmap's doubling allocation strategy itself, keeping open the discussion on the most effective approach for a comprehensive solution.
Nov 16, 07:47 The latest reply acknowledges the proposer's findings regarding kernel LRU list lock contention and confirms the understanding of glibc's adaptive malloc threshold behavior. The message is truncated, suggesting further discussion or review might follow.
archive ↗
No threads match — try clearing filters.