Skip to content

Agent friendly checks: make review conventions mechanically checkable - #9748

Open
sangbida wants to merge 12 commits into
ElementsProject:masterfrom
sangbida:agent-friendly-checks
Open

sangbida wants to merge 12 commits into
ElementsProject:masterfrom
sangbida:agent-friendly-checks

Conversation

@sangbida

Copy link
Copy Markdown
Collaborator

Motivation

More code is being written with coding agents, and agents infer conventions
from the code already in the repository. Review feedback which exists only
in documentation or PR discussions is easy to miss and has to be enforced
repeatedly by humans.

This turns a set of recurring CLN review comments into deterministic checks.

The rules apply to all contributors, but are designed to give agents useful
feedback while they build and test their changes.
Existing code is grandfathered: source checks apply only to lines or
definitions added by the branch.

Source checks

This adds checks for:

  • pytest.mark.xfail without strict=True;
  • default: added to a switch over an enum;
  • comments which narrate the change, describe removed code, cite a ticket or
    review, or point to a line number;
  • comments longer than 4 lines inside code or 8 lines at the left margin;
  • new non-static C functions which nothing calls;
  • take() of objects allocated from tmpctx or another non-NULL parent.
    BOLT quotes, tutorial comments and generated wire helpers have the necessary
    exceptions.

Commit checks

Commit messages are checked for:

  • wrapped, readable prose and short paragraphs;
  • a blank line after the subject;
  • subjects in subsystem: intent form;
  • references to a PR, its review or another commit by relative position;
  • markdown headings.
    The old commitlint prefix allowlist is removed because it rejected valid CLN
    subsystems and needed updating for every new plugin or directory.

Per-commit verification

make check-commits checks every commit in a branch by running:

  • the build;
  • make check-source;
  • unit tests;
  • integration test files touched by that commit.
    make check-commits-full runs the complete integration suite on every
    commit.
    An opt-in GitHub workflow provides the same per-commit verification for a PR.
    Maintainers can enable it with the per-commit-checks label, the
    PER_COMMIT_CHECKS repository variable or workflow_dispatch. Rust is
    disabled when the PR does not touch Rust inputs.

Changelog validation

The changelog CI check now accepts only entries which changelog.py will
actually collect, preventing malformed entries from passing CI and then
disappearing from the release notes.

Changelog-None

sangbida and others added 12 commits November 19, 2026 10:07
CI accepted malformed Changelog- lines which changelog.py ignored.
Match the parser's accepted prefixes so CI and the release notes agree.

Changelog-None
Co-authored-by: Cursor <cursoragent@cursor.com>
A non-strict xfail also passes when the test unexpectedly succeeds.
Reject added pytest.mark.xfail decorators without strict=True.
Co-authored-by: Cursor <cursoragent@cursor.com>
A default label hides the compiler warning when an enum gains a value.
Reject added defaults in switches whose cases are enum values.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add make check-commits to build each commit and run check-source, unit
tests and touched integration tests.  check-commits-full runs all
integration tests instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add a workflow which maintainers can enable with a repository variable,
a PR label or workflow_dispatch.  It enables Rust only when the PR
touches Rust inputs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Comments should describe the code a reader can see.  Reject added
comments which narrate the change, describe removed code, cite a ticket
or review, or point to a line number.
Co-authored-by: Cursor <cursoragent@cursor.com>
Limit comments inside code to 4 lines and comments at the left margin
to 8.  BOLT quotes and tutorial comments are exempt.
Co-authored-by: Cursor <cursoragent@cursor.com>
Check each new commit for wrapped text, short paragraphs and a blank
line after the subject.  Reject markdown headings and references to a
PR, its review or another commit by relative position.
Co-authored-by: Cursor <cursoragent@cursor.com>
Reject a new non-static function when no tracked C file calls it.
Calls from tests and later commits in the branch count; generated wire
helpers are exempt.
Co-authored-by: Cursor <cursoragent@cursor.com>
An object allocated only to be handed over with take() should have no
other owner.  Reject added take() calls which allocate from tmpctx or
pass a non-NULL parent to tal or towire helpers.
Co-authored-by: Cursor <cursoragent@cursor.com>
Require commit subjects to use the "subsystem: intent" form from the
style guide.  Preserve exceptions for fixups, releases and reverts.
Co-authored-by: Cursor <cursoragent@cursor.com>
The prefix allowlist rejects valid subsystems and requires constant
maintenance.  check-source now checks subject shape without an
allowlist.
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread tools/check-commits.sh
continue
fi

git checkout -q --detach "$sha" || exit 1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check-commits.sh doesnt update submodules on checkout

Comment thread tools/check-commits.sh
echo "[$n/$total] $(git log -1 --format='%h %s' "$sha")"

config=$(git hash-object config.vars 2> /dev/null || echo unconfigured)
key="$(git rev-parse "$sha^{tree}") $(git rev-parse -q --verify "$sha^^{tree}" || echo root) $pytest_mode $config"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the check-commits cache key ignores the commit message

@@ -0,0 +1,99 @@
#!/usr/bin/env bash

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check-xfail-strict.sh can be replaced by one config line? add xfail_strict = true to tests/pytest.ini

for case in CASE.finditer(code, body[0], body[1]):
label = case.group(1)
if innermost(bodies, case.start()) != body:
continue

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A switch is flagged as soon as any case label is an enumerator. On master it flags lightningd/peer_htlcs.c at line 2864 (3 of +- 25 onion_wire values) and wire/peer_wire.c at line 74 (is_oversized_gossip_msg, 3 of +-40 peer_wire values)

Comment thread Makefile

# Only lines added since BASE_REF are checked (doc/Makefile sets the
# default): see tools/added-lines.sh.
check-xfail-strict:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default BASE_REF=master gives false positives with a stale local master? maybe lets use BASE_REF ?= origin/master

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can have the same staleness issue with either local master or origin/master. Also, not everyone names their remote origin, and origin is usually the contributor’s fork rather than the upstream ElementsProject/lightning repository.

Comment thread tools/added-lines.sh
header = 0
split($3, range, ",")
line = substr(range[1], 2) + 0
next

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

??? maybe a softer limit, or a warning instead of a failure, may fit better?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you're referring to here?

SUBJECT = re.compile(r'((fixup|squash|amend)! )*(\S+ ){0,2}\S+: \S|v\d+\.\d+\S*$|Revert "')
UNSEEN = re.compile(r'\bthis (pr|mr|pull request)\b|\breview(er)? (comments?|feedback)\b'
r'|\bper review\b|\bas (requested|discussed)\b', re.I)
BY_POSITION = re.compile(r'\b(previous|last|earlier|prior|preceding|parent) (commit|patch)(es|s)?\b', re.I)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the the commit message rules reject wording maintainers currently accept?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you referring to the by position rule? I think as maintainers, it's hard to pick up on subtleties like this every time on review and I'm personally unlikely to send back a PR if the only change requested in a commit message change which mentions the previous commit.

Ideally commits should stand alone and someone reading the commit should be able to understand what is going on without traversing the commits in a PR. PRs are ephemeral because once they are merged into main it's hard to tell which commits were a part of which PR.

Also, can we REALLY trust github to be around forever? 👀

I added this rule because more and more I find myself and see others writing more and more commit messages using AI, which is totally fine, but this way the LLMs (and also humans) can hopefully write better commit messages.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants