Skip to content

[python] Speed up BLOB fallback reads for disjoint row ranges - #11266

Open
XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:fix/python-blob-disjoint-20261119
Open

XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:fix/python-blob-disjoint-20261119

Conversation

@XiaoHongbo-Hope

Copy link
Copy Markdown
Contributor

Purpose

When BLOB files in a logical read range have disjoint row IDs, different sequence numbers do not represent competing versions. The Python fallback reader nevertheless checks every file state on every batch and searches sequence dictionaries for each output row.

Detect disjoint file ranges, visit them in row order with a forward-only active cursor, and resolve their values through one group. Keep the existing fallback behavior for overlapping files and preserve selected ranges, deletion vectors, gaps, NULLs, and placeholders. This addresses the Python analogue of #11225 without changing the file format or Java reader.

Tests

  • 115 tests passed, 1 skipped, 7 large-data tests deselected across the BLOB table, deletion-vector, and batch-reader suites.
  • The new 128-file regression fails on the original reader with 16,384 state checks; the patched reader stays within 256 checks and returns identical rows.
  • Coverage includes selected gaps, placeholders, explicit NULLs, deletion vectors, and existing descriptor/update/rolling-file reads.
  • Flake8 with the repository configuration and git diff --check passed.

@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as draft November 19, 2026 04:01
@XiaoHongbo-Hope
XiaoHongbo-Hope marked this pull request as ready for review November 19, 2026 04:26
@XiaoHongbo-Hope XiaoHongbo-Hope changed the title [python] Avoid redundant fallback work for disjoint BLOB files [python] Speed up BLOB fallback reads for disjoint row ranges Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant