Repository navigation
fix(file): normalize fixed-size struct list row counts - #9999
Merged
Merged
Conversation
wjones127
approved these changes
Oct 19, 2026
…20261030 # Conflicts: # rust/lance-file/src/reader.rs
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
Scalar and FixedSizeList<Struct> columns now round-trip together without changing stored encodings. Dense normalization rejects non-divisible row counts, while sparse normalization retains its existing semantics. The file crate suite and additional nested/null, multi-page projection checks pass.
Xuanwo
approved these changes
Oct 30, 2026
Xuanwo
left a comment
Member
There was a problem hiding this comment.
Verified the current revision after merging main. The fixed-size-list row scale now covers stable 2.2 Structural pages and dense 2.3 Planar pages while preserving sparse page-local normalization. Non-divisible dense counts return an invalid-input error, and the Python round trip plus full current-head CI pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A valid table with a scalar column and a two-item
FixedSizeList<Struct>writes successfully in the default 2.2 format, butto_table()rejects it withcannot read columns of differing lengths together (id=1, points=2). We expect the original one-row table, as the read guide describes.We scale dense page counts by the structural list dimensions during projection validation and preserve sparse-page normalization.
The Python reproducer and all 260 Rust file tests pass, including dense 2.2/2.3 and sparse 2.3 regression cases.
Workspace Clippy is unverified; crate Clippy without
--no-depsfailed on existinglance-iodead-code warnings.— ChipLab