Skip to content

feat(wal): make the per-follower replication window configurable - #845

Merged
jja725 merged 2 commits into
lance-format:mainfrom
jja725:codex/wal-inflight-options-20261110
Nov 17, 2026
Merged

jja725 merged 2 commits into
lance-format:mainfrom
jja725:codex/wal-inflight-options-20261110

Conversation

@jja725

@jja725 jja725 commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

Problem

Runtime workers always use a 16-message follower window, so replication experiments cannot vary the amount of outstanding AppendEntries traffic independently of durable policy. Refs #378.

Fix

Add runtime.raft_max_inflight_msgs (1–256, default 16) and apply it to every static voter before group workers start. Existing byte credits and uncommitted-byte limits remain authoritative. Changing the window on restart does not change the persisted policy.

Verification

  • All 30 runtime tests passed with service, including blocked-sync elections and recovery.
  • New tests with windows 1 and 3 hold real follower replies, verify the send bound independently for two followers and two groups, then resume durable commits and reopen with a different window. Invalid windows reject startup without retaining the directory.
  • Clippy with service (--all-targets -- -D warnings), workspace formatting and git diff --check passed.
  • No production throughput claim; the default window is unchanged.

@beinan beinan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed startup validation and application to every static voter before worker startup, including raft-rs capacity retention across progress resets. The per-follower/per-group tests exercise held replies, resumed durable commits, and reopening with a different window. Existing byte admission limits and the default window remain intact. No blocking findings. All three CI workflows passed for this commit.

@jja725
jja725 marked this pull request as ready for review November 17, 2026 02:05

@beinan beinan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current merge with main. The validated per-follower window is applied before workers start, preserves the default of 16, and leaves byte admission and quorum persistence boundaries unchanged. The tests cover bounded sends, resumption on durable replies, restart with a new window, and invalid values. Current-head WAL CI has passed; remaining CI is still running.

@jja725
jja725 merged commit f1ed7ca into lance-format:main Nov 17, 2026
12 checks passed
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants