feat: add AI attribution governance (forbid known AI tool signatures) - #456
Conversation
Introduce a new validation subsystem that lets projects enforce their AI contribution policy at the commit-message level. This feature is motivated by the ongoing industry-wide discussion around AI disclosure in open source (CPython, Linux kernel, VS Code, Apache, Fedora, etc.) and requires no external dependencies. Configuration (under [commit]): ai_attribution = "forbid" | "require" | "ignore" (default: ignore) ai_trailer_style = "assisted-by" | "co-authored-by" (default: assisted-by) Key components: - commit_check/ai_signatures.py — curated database of known AI tool signatures (Claude Code, Copilot, Codex, Gemini, Cursor, Devin, Aider, Windsurf, Tabby, and generic AI patterns) - AiAttributionValidator — three-mode policy engine - AiTrailerStyleValidator — enforces Linux kernel Assisted-by or GitHub Co-authored-by style - Full CLI, API, env-var, and TOML config integration - 52 new tests covering all modes and detection patterns
✅ Deploy Preview for commit-check ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
github-actions
Bot
added
enhancement
📝 WalkthroughWalkthroughAdds AI attribution enforcement for commit messages. Introduces AI signature data and detection helpers, wires a new ChangesAI Attribution Feature
Estimated code review effort: 3 (Moderate) | ~30 minutes Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #456 +/- ##
==========================================
+ Coverage 96.89% 97.10% +0.21%
==========================================
Files 10 12 +2
Lines 1094 1176 +82
==========================================
+ Hits 1060 1142 +82
Misses 34 34 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (8)
tests/ai_signatures_test.py (1)
206-210: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUnused loop variable
regex.Ruff (B007) flags
regexas unused within the loop body.♻️ Proposed fix
- for regex, tool_name, desc in ALL_PATTERNS: + for _regex, tool_name, desc in ALL_PATTERNS: assert desc, f"Pattern for {tool_name} is missing a description"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/ai_signatures_test.py` around lines 206 - 210, The loop in test_all_patterns_have_description uses ALL_PATTERNS but never reads the regex value, so Ruff flags the unused variable. Update the iteration in test_all_patterns_have_description to ignore that tuple element (for example by using a placeholder name) while keeping tool_name and desc checks unchanged, so the intent stays clear and the B007 warning is removed.Source: Linters/SAST tools
commit_check/rule_builder.py (1)
283-307: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winConfirm
ai_trailer_stylerule should be built underforbidtoo.This rule is built whenever
ai_attribution != "ignore", including"forbid". But per theAiTrailerStyleValidatordocstring in engine.py, it's meant as "a companion toai_attribution = 'require'". Underforbid, any detected AI signature is already rejected byAiAttributionValidator; additionally running the trailer-style check on the same message could produce a second, redundant failure for what is effectively a single underlying violation.If this is intentional (e.g., to also surface style feedback under
forbid), consider ignoring; otherwise restrict rule construction topolicy == "require".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@commit_check/rule_builder.py` around lines 283 - 307, The _build_ai_trailer_style_rule method currently creates the ai_trailer_style ValidationRule for any ai_attribution value except ignore, which includes forbid and can cause redundant failures. Update the rule construction logic in _build_ai_trailer_style_rule so it only builds the trailer-style rule when ai_attribution is require, matching the AiTrailerStyleValidator intent in engine.py; keep the existing ai_trailer_style validation and ValidationRule wiring unchanged otherwise.commit_check/rules_catalog.py (1)
107-118: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUnused
{reason}/{suggestion}templates.Per the
AiAttributionValidator/AiTrailerStyleValidatorimplementations in engine.py, failures are recorded with fully-formederror/suggeststrings passed directly to_record_failure(...), not viacatalog_entry.error.format(reason=...)orcatalog_entry.suggest.format(suggestion=...). These templated placeholders appear to be dead code that could mislead readers into thinking they're filled in.Consider either wiring these templates into the validators'
_record_failurecalls, or simplifying the catalog entries to static text if the templating is unused.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@commit_check/rules_catalog.py` around lines 107 - 118, The ai_attribution and ai_trailer_style catalog entries are using dead `{reason}` and `{suggestion}` templates that the validators do not populate. Update the `AiAttributionValidator` and `AiTrailerStyleValidator` paths in engine.py to either pass formatted values into `_record_failure(...)` and consume the templates, or simplify the corresponding `RuleCatalogEntry` `error`/`suggest` fields in `rules_catalog.py` to static text. Keep the catalog and validator behavior aligned so the `check` names and failure messages are consistent.tests/rule_builder_test.py (1)
375-491: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider adding a test for an invalid
ai_attributionvalue.Given the validation gap flagged in
rule_builder.py(_build_ai_attribution_ruleonly special-cases"ignore"), a test asserting behavior for an unrecognized value (e.g."typo") would help pin down the intended contract once that's addressed.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/rule_builder_test.py` around lines 375 - 491, Add a test in TestAiAttributionRuleBuilder for an invalid ai_attribution value such as "typo" to define the expected contract for RuleBuilder._build_ai_attribution_rule. Set up a commit config with the bad value, build the rule via _build_ai_attribution_rule, and assert the intended outcome explicitly (for example, reject/return None or raise) so the behavior is pinned down alongside the existing ignore/forbid/require cases.commit_check/engine.py (2)
785-802: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate
_record_failureimplementation across both validators.
AiAttributionValidator._record_failureandAiTrailerStyleValidator._record_failureare byte-for-byte identical. Consider hoisting this intoBaseValidator(which already has a similarly-shaped_print_failure) to avoid drift between the two copies.♻️ Proposed consolidation
class BaseValidator(ABC): ... + def _record_failure(self, value: str, error: str, suggest: str) -> None: + """Record a failure with dynamic error/suggest messages.""" + self._last_failure = { + "check": self.rule.check, + "value": value, + "error": error, + "suggest": suggest, + } + if not self._suppress_output: + rule_dict = self.rule.to_dict() + from commit_check.util import _print_failure + + _print_failure( + rule_dict, + value, + no_banner=self._no_banner, + compact=self._compact, + )Then remove the duplicated method bodies from both
AiAttributionValidatorandAiTrailerStyleValidator.Also applies to: 855-872
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@commit_check/engine.py` around lines 785 - 802, The `_record_failure` logic is duplicated in both `AiAttributionValidator` and `AiTrailerStyleValidator`, so consolidate it into `BaseValidator` alongside `_print_failure`-style handling. Move the shared implementation into `BaseValidator._record_failure` using the existing `self.rule`, `self._last_failure`, `self._suppress_output`, and `_print_failure` flow, then remove the copied method bodies from both validator classes so they inherit the shared behavior.
728-783: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReduce cognitive complexity of
AiAttributionValidator.validate().SonarCloud flags this method's cognitive complexity at 21 vs. the allowed 15. Consider extracting the
forbidandrequirebranches into private helper methods (e.g._validate_forbid,_validate_require) to improve readability and satisfy the linter.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@commit_check/engine.py` around lines 728 - 783, The validate method in AiAttributionValidator is too complex and should be simplified by extracting the policy-specific logic into private helpers. Move the "forbid" and "require" branches out of validate() into dedicated methods such as _validate_forbid and _validate_require, and have validate() only handle the early exits, policy dispatch, and result return. Keep the existing behavior and reuse the current helpers like detect_ai_signatures, find_co_authored_by_ai, and find_assisted_by_trailers inside the new methods.Source: Linters/SAST tools
tests/engine_test.py (2)
1630-1630: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDuplicated string literals flagged by static analysis.
SonarCloud flags the literal
"feat: add feature\n\nCo-authored-by: Claude <noreply@anthropic.com>"(and the Assisted-by equivalent) as duplicated several times across tests. Consider extracting shared fixture strings to module-level constants.Also applies to: 1643-1643, 1699-1699, 1741-1741, 1754-1754, 1798-1798, 1837-1837
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/engine_test.py` at line 1630, The test strings used in the commit-message assertions are duplicated across multiple cases, so extract the shared `"feat: add feature\n\nCo-authored-by: Claude <noreply@anthropic.com>"` and Assisted-by variants into module-level constants in the test module. Update the affected test functions that reference these literals to use the shared constants instead, keeping the existing assertions and commit-message behavior unchanged.Source: Linters/SAST tools
1619-1773: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winMissing test for signature-without-trailer under
requirepolicy.None of the
TestAiAttributionValidatortests cover a message with a detected AI signature via a non-trailerbody_markerpattern and noCo-authored-by/Assisted-bytrailer at all underrequirepolicy. This is exactly the gap flagged inengine.py'sAiAttributionValidator.validate()(require branch) — currently that scenario incorrectly PASSes. Adding a test would have caught it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/engine_test.py` around lines 1619 - 1773, Add a missing AiAttributionValidator test for the require policy covering a commit that contains an AI signature detected via a body_marker pattern but no Co-authored-by or Assisted-by trailer. Use TestAiAttributionValidator and AiAttributionValidator.validate with ValidationRule(check="ai_attribution", value="require", allowed=[...]) to assert this message fails, since the current require branch in validate() incorrectly passes that case.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@commit_check/ai_signatures.py`:
- Around line 227-232: The generic Co-authored-by AI trailer pattern in
ai_signatures.py is too strict and only matches bare model names, so it misses
documented suffix forms like claude-sonnet or gpt-4. Update the _trailer pattern
passed for Co-authored-by so it matches model-name prefixes plus optional suffix
characters before the email/address part, while still allowing the trailer to
terminate correctly; keep the change localized to the Co-authored-by signature
entry and preserve the existing AI model family coverage.
- Around line 262-306: The signature kind is being reconstructed incorrectly
from the regex prefix, which mislabels some body markers as trailers. Update the
flattened registry in ALL_PATTERNS to preserve AiSignaturePattern.kind alongside
the regex, tool name, and description, then use that stored kind in
detect_ai_signatures instead of checking regex.pattern.startswith("^"). Also
adjust the tuple unpacking in has_ai_signature, find_co_authored_by_ai, and
find_assisted_by_trailers to match the new ALL_PATTERNS shape.
In `@commit_check/engine.py`:
- Around line 753-781: The require path in the commit validation logic only
checks for mismatched trailer styles and can incorrectly PASS when AI signatures
are present but no valid trailer exists. Update the require branch in
`commit_check.engine`’s validation flow to explicitly fail whenever
`detect_ai_signatures` returns matches but neither `find_co_authored_by_ai` nor
`find_assisted_by_trailers` finds the configured trailer style, and keep the
existing style-mismatch failures for the wrong trailer type. Add a clear failure
via `_record_failure` using the same `ValidationResult.FAIL` path so `require`
truly enforces that a correctly styled AI attribution trailer is present.
- Around line 716-874: Avoid running both AI validators on the same commit:
AiAttributionValidator already checks trailer style when ai_attribution is
"require", so AiTrailerStyleValidator should not also fire for that policy.
Update the validation flow in AiAttributionValidator and AiTrailerStyleValidator
so the style-specific check is only active when needed, and adjust the request
path in api.py/main.py to avoid requesting ai_trailer_style for redundant cases
or centralize the style enforcement in one validator to prevent duplicate FAILs
and double scanning.
In `@commit_check/rule_builder.py`:
- Around line 257-281: Validate the ai_attribution policy in
_build_ai_attribution_rule before creating the ValidationRule: right now only
"ignore" is handled and typos can slip through silently. In
RuleBuilder._build_ai_attribution_rule, check the resolved value from
self.commit_config.get("ai_attribution", DEFAULT_AI_ATTRIBUTION) against
{"ignore", "forbid", "require"} and fail loudly or raise an error for anything
else, so invalid config/CLI input does not produce a no-op validator. Keep the
existing handling for the valid modes and the trailer style lookup unchanged.
In `@tests/ai_signatures_test.py`:
- Around line 115-119: The assertion in test_all_patterns_compile is
tautological and never validates the regex result. Update the check in
test_all_patterns_compile so it actually exercises regex.search or another
compilation/validation path for each entry in ALL_PATTERNS, and remove the
unconditional or True. Keep the existing tool_name and desc context in the
failure message so a bad pattern is still easy to identify.
---
Nitpick comments:
In `@commit_check/engine.py`:
- Around line 785-802: The `_record_failure` logic is duplicated in both
`AiAttributionValidator` and `AiTrailerStyleValidator`, so consolidate it into
`BaseValidator` alongside `_print_failure`-style handling. Move the shared
implementation into `BaseValidator._record_failure` using the existing
`self.rule`, `self._last_failure`, `self._suppress_output`, and `_print_failure`
flow, then remove the copied method bodies from both validator classes so they
inherit the shared behavior.
- Around line 728-783: The validate method in AiAttributionValidator is too
complex and should be simplified by extracting the policy-specific logic into
private helpers. Move the "forbid" and "require" branches out of validate() into
dedicated methods such as _validate_forbid and _validate_require, and have
validate() only handle the early exits, policy dispatch, and result return. Keep
the existing behavior and reuse the current helpers like detect_ai_signatures,
find_co_authored_by_ai, and find_assisted_by_trailers inside the new methods.
In `@commit_check/rule_builder.py`:
- Around line 283-307: The _build_ai_trailer_style_rule method currently creates
the ai_trailer_style ValidationRule for any ai_attribution value except ignore,
which includes forbid and can cause redundant failures. Update the rule
construction logic in _build_ai_trailer_style_rule so it only builds the
trailer-style rule when ai_attribution is require, matching the
AiTrailerStyleValidator intent in engine.py; keep the existing ai_trailer_style
validation and ValidationRule wiring unchanged otherwise.
In `@commit_check/rules_catalog.py`:
- Around line 107-118: The ai_attribution and ai_trailer_style catalog entries
are using dead `{reason}` and `{suggestion}` templates that the validators do
not populate. Update the `AiAttributionValidator` and `AiTrailerStyleValidator`
paths in engine.py to either pass formatted values into `_record_failure(...)`
and consume the templates, or simplify the corresponding `RuleCatalogEntry`
`error`/`suggest` fields in `rules_catalog.py` to static text. Keep the catalog
and validator behavior aligned so the `check` names and failure messages are
consistent.
In `@tests/ai_signatures_test.py`:
- Around line 206-210: The loop in test_all_patterns_have_description uses
ALL_PATTERNS but never reads the regex value, so Ruff flags the unused variable.
Update the iteration in test_all_patterns_have_description to ignore that tuple
element (for example by using a placeholder name) while keeping tool_name and
desc checks unchanged, so the intent stays clear and the B007 warning is
removed.
In `@tests/engine_test.py`:
- Line 1630: The test strings used in the commit-message assertions are
duplicated across multiple cases, so extract the shared `"feat: add
feature\n\nCo-authored-by: Claude <noreply@anthropic.com>"` and Assisted-by
variants into module-level constants in the test module. Update the affected
test functions that reference these literals to use the shared constants
instead, keeping the existing assertions and commit-message behavior unchanged.
- Around line 1619-1773: Add a missing AiAttributionValidator test for the
require policy covering a commit that contains an AI signature detected via a
body_marker pattern but no Co-authored-by or Assisted-by trailer. Use
TestAiAttributionValidator and AiAttributionValidator.validate with
ValidationRule(check="ai_attribution", value="require", allowed=[...]) to assert
this message fails, since the current require branch in validate() incorrectly
passes that case.
In `@tests/rule_builder_test.py`:
- Around line 375-491: Add a test in TestAiAttributionRuleBuilder for an invalid
ai_attribution value such as "typo" to define the expected contract for
RuleBuilder._build_ai_attribution_rule. Set up a commit config with the bad
value, build the rule via _build_ai_attribution_rule, and assert the intended
outcome explicitly (for example, reject/return None or raise) so the behavior is
pinned down alongside the existing ignore/forbid/require cases.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6d8da658-2676-4758-af02-62600c0a929c
📒 Files selected for processing (11)
commit_check/__init__.pycommit_check/ai_signatures.pycommit_check/api.pycommit_check/config_merger.pycommit_check/engine.pycommit_check/main.pycommit_check/rule_builder.pycommit_check/rules_catalog.pytests/ai_signatures_test.pytests/engine_test.pytests/rule_builder_test.py
| def _build_ai_attribution_rule( | ||
| self, catalog_entry: RuleCatalogEntry | ||
| ) -> ValidationRule | None: | ||
| """Build AI attribution validation rule. | ||
|
|
||
| Three modes: | ||
| * ``"forbid"`` — reject any commit with AI tool signatures | ||
| * ``"require"`` — if AI signatures present, must use preferred style | ||
| * ``"ignore"`` — no validation (default, returns None) | ||
| """ | ||
| policy = self.commit_config.get("ai_attribution", DEFAULT_AI_ATTRIBUTION) | ||
| if policy == "ignore": | ||
| return None | ||
|
|
||
| trailer_style = self.commit_config.get( | ||
| "ai_trailer_style", DEFAULT_AI_TRAILER_STYLE | ||
| ) | ||
|
|
||
| return ValidationRule( | ||
| check=catalog_entry.check, | ||
| value=policy, | ||
| error=catalog_entry.error or "", | ||
| suggest=catalog_entry.suggest or "", | ||
| allowed=[trailer_style], | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reject invalid ai_attribution values before building the rule
ai_attribution only special-cases "ignore" here; a typo from config/CLI is passed through and can make the validator silently do nothing. Validate the policy against {"ignore", "forbid", "require"} or fail loudly before constructing the rule.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@commit_check/rule_builder.py` around lines 257 - 281, Validate the
ai_attribution policy in _build_ai_attribution_rule before creating the
ValidationRule: right now only "ignore" is handled and typos can slip through
silently. In RuleBuilder._build_ai_attribution_rule, check the resolved value
from self.commit_config.get("ai_attribution", DEFAULT_AI_ATTRIBUTION) against
{"ignore", "forbid", "require"} and fail loudly or raise an error for anything
else, so invalid config/CLI input does not produce a no-op validator. Keep the
existing handling for the valid modes and the trailer style lookup unchanged.
Merging this PR will not alter performance
Performance Changes
Comparing 
Footnotes
|
P0 fixes: - Template leak in text output: _record_failure now passes dynamic error/suggest into the rule_dict before printing. - Dual validator conflict: ai_attribution handles only 'forbid'; ai_trailer_style handles only 'require'. One policy = one check. P1 fixes: - Kernel Assisted-by format now accepts optional trailing tool list. - Generic model names now match claude-sonnet-4, gpt-4-turbo, etc. - ALL_PATTERNS tuples include 'kind' field (fixes body_marker misclassification as trailer). - find_co_authored_by_ai deduplicates via seen set. - Claude regex anchors to noreply@anthropic.com / GitHub noreply to avoid false positives with human co-authors named Claude. - Copilot regex anchored to its specific GitHub noreply address. - Added aider '(aider)' author suffix pattern. - Removed dead github-actions[bot] pattern. - Updated tests: capsys output checks, human-name false-positive regression tests, kernel format fixtures.
Simplify AI attribution to a single binary choice: ai_attribution = "ignore" | "forbid" (default: "ignore") Rationale: - Different AI tools generate different trailer styles (Co-authored-by vs Assisted-by vs future formats). Enforcing a single style is not commit-check's job -- 'forbid' already rejects all known formats. - 'require' mode gave a false sense of security (can't detect undisclosed AI usage) and required users to pick a style side. Changes: - Remove AiTrailerStyleValidator class and all its references - Remove ai_trailer_style TOML key, CLI arg, env var, and default - Remove require from --ai-attribution choices (only ignore|forbid) - Remove ai_trailer_style from rules_catalog, config_merger, api, main.py, engine VALIDATOR_MAP, and all related tests - AiAttributionValidator now handles only forbid (single responsibility) - Simplify rule_builder: only builds ai_attribution rule for forbid
- configuration.rst: add ai_attribution to example config, env var table, CLI mapping, and options reference table - README.rst: mention AI attribution in overview and add config example with ai_attribution = "forbid" - what-is-new.rst: add v2.10.0 entry describing the AI attribution governance feature and the full list of detected tools
github-actions
Bot
added
the
documentation
shenxianpeng
changed the title
Follow imperatives.py pattern — ai_signatures_data.py holds tool definitions; ai_signatures.py holds detection API only.
shenxianpeng
removed
documentation
The generic AI model-name pattern matched any Co-authored-by trailer whose value started with claude/gpt/gemini plus any email, so a human co-author with a bare first name such as "Claude" would be flagged incorrectly. Require a hyphenated model suffix (e.g. claude-sonnet-4) so bare human names are never matched while real model identifiers still are. Also remove the now-unused find_co_authored_by_ai and find_assisted_by_trailers helpers left over from the dropped trailer-style mode, fix the stale AiAttributionValidator docstring, and add regression tests covering human-name false positives and model-identifier detection.
github-actions
Bot
added
documentation
|
shenxianpeng
removed
documentation
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
commit_check/rule_builder.py (1)
254-271: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winInvalid/typo
ai_attributionconfig values silently behave as "ignore".Any value other than
"forbid"(e.g. a typo like"forbidd"fromcchk.tomlor an env var) falls through toreturn None, silently disabling the check with no warning. The CLI mitigates this viachoices=["ignore", "forbid"], but TOML/env config paths aren't similarly constrained. A misconfigured project believes AI-attribution enforcement is active when it is not.This is the same class of issue flagged in an earlier review (previously scoped to the 3-value ignore/forbid/require model); it still applies to the now-simplified binary model.
🐛 Proposed fix to fail loudly on invalid policy values
def _build_ai_attribution_rule( self, catalog_entry: RuleCatalogEntry ) -> ValidationRule | None: """Build AI attribution validation rule. Only active when policy is ``"forbid"`` — rejects any commit with known AI tool signatures. """ policy = self.commit_config.get("ai_attribution", DEFAULT_AI_ATTRIBUTION) - if policy != "forbid": + if policy not in ("ignore", "forbid"): + raise ValueError( + f"Invalid ai_attribution policy {policy!r}; expected 'ignore' or 'forbid'" + ) + if policy != "forbid": return None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@commit_check/rule_builder.py` around lines 254 - 271, The ai_attribution policy handling in _build_ai_attribution_rule currently treats any non-"forbid" value as disabled, so typos from commit_config can silently skip enforcement. Update the policy lookup and ValidationRule creation path to explicitly accept only the supported values ("ignore" and "forbid"), and make invalid values fail loudly instead of returning None. Use _build_ai_attribution_rule, commit_config, and DEFAULT_AI_ATTRIBUTION as the key places to adjust validation and error handling.tests/ai_signatures_test.py (1)
177-186: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winTest can pass vacuously if body-marker detection breaks.
If
detect_ai_signaturesreturns an empty list (e.g., the body-marker pattern regresses), thefor r in result:loop never executes and the test passes without exercising the assertion at all — the same class of issue as the previously-fixed tautological assertion.🐛 Proposed fix
def test_kind_field_correct_for_body_marker(self): """Body markers have kind='body_marker', not 'trailer'.""" message = "feat: add feature\n\nGenerated by AI" result = detect_ai_signatures(message) + assert len(result) >= 1, "Expected at least one match for 'Generated by AI'" for r in result: if r["description"].startswith("``Generated by AI"): assert r["kind"] == "body_marker", ( f"Expected body_marker, got {r['kind']}" )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/ai_signatures_test.py` around lines 177 - 186, The test in detect_ai_signatures is vulnerable to vacuous success because it only asserts inside the loop over result, so an empty list never exercises the body-marker check. Update test_kind_field_correct_for_body_marker to first assert that detect_ai_signatures(message) returns at least one matching body-marker entry (or otherwise fail when none is found), then verify that the matching item’s kind is body_marker using the existing detect_ai_signatures helper and the description prefix check.
🧹 Nitpick comments (1)
tests/ai_signatures_test.py (1)
271-271: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRename unused loop variables per Ruff hint.
Static analysis flags
regexandkindas unused in this loop body.🧹 Proposed fix
- for regex, tool_name, desc, kind in ALL_PATTERNS: + for _regex, tool_name, desc, _kind in ALL_PATTERNS: assert desc, f"Pattern for {tool_name} is missing a description"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/ai_signatures_test.py` at line 271, The loop over ALL_PATTERNS in the AI signatures test has unused bindings for regex and kind, which Ruff flags. Update the tuple unpacking in the relevant test loop to use placeholder names for the unused values while keeping tool_name and desc as-is, so the intent is clear and the warning is removed.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@commit_check/ai_signatures_data.py`:
- Around line 159-169: The DEVIN signature is too broad because the current
Co-authored-by matcher in ai_signatures_data.py accepts any person named Devin,
so tighten the KnownAiTool pattern to match Devin’s specific bot identity/email
instead of a bare name. Update the DEVIN entry and its _trailer matcher so only
the intended bot co-author format is recognized, and ensure human co-authors
named Devin no longer match.
---
Duplicate comments:
In `@commit_check/rule_builder.py`:
- Around line 254-271: The ai_attribution policy handling in
_build_ai_attribution_rule currently treats any non-"forbid" value as disabled,
so typos from commit_config can silently skip enforcement. Update the policy
lookup and ValidationRule creation path to explicitly accept only the supported
values ("ignore" and "forbid"), and make invalid values fail loudly instead of
returning None. Use _build_ai_attribution_rule, commit_config, and
DEFAULT_AI_ATTRIBUTION as the key places to adjust validation and error
handling.
In `@tests/ai_signatures_test.py`:
- Around line 177-186: The test in detect_ai_signatures is vulnerable to vacuous
success because it only asserts inside the loop over result, so an empty list
never exercises the body-marker check. Update
test_kind_field_correct_for_body_marker to first assert that
detect_ai_signatures(message) returns at least one matching body-marker entry
(or otherwise fail when none is found), then verify that the matching item’s
kind is body_marker using the existing detect_ai_signatures helper and the
description prefix check.
---
Nitpick comments:
In `@tests/ai_signatures_test.py`:
- Line 271: The loop over ALL_PATTERNS in the AI signatures test has unused
bindings for regex and kind, which Ruff flags. Update the tuple unpacking in the
relevant test loop to use placeholder names for the unused values while keeping
tool_name and desc as-is, so the intent is clear and the warning is removed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ba564f90-942a-4ef0-9300-09c0c4d01e29
📒 Files selected for processing (15)
README.rstcommit_check/__init__.pycommit_check/ai_signatures.pycommit_check/ai_signatures_data.pycommit_check/api.pycommit_check/config_merger.pycommit_check/engine.pycommit_check/main.pycommit_check/rule_builder.pycommit_check/rules_catalog.pydocs/configuration.rstdocs/what-is-new.rsttests/ai_signatures_test.pytests/engine_test.pytests/rule_builder_test.py
💤 Files with no reviewable changes (3)
- commit_check/api.py
- commit_check/config_merger.py
- tests/engine_test.py
✅ Files skipped from review due to trivial changes (2)
- docs/configuration.rst
- docs/what-is-new.rst
| # --- Devin --- | ||
| DEVIN = KnownAiTool( | ||
| name="Devin", | ||
| patterns=[ | ||
| _trailer( | ||
| "Co-authored-by", | ||
| r"Devin\s*(?:<[^>]*>)?", | ||
| "``Co-authored-by: Devin`` trailer", | ||
| ), | ||
| ], | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Devin AI Cognition commit co-authored-by bot email address
💡 Result:
Devin AI, developed by Cognition, uses a Co-authored-by trailer in its git commit messages to provide attribution for changes made by the AI agent [1][2]. Regarding the email address associated with this trailer, official documentation for Devin notes a correction in its changelog: the system was updated to use the correct GitHub App bot email address instead of the previously used noreply@cognition.ai address [3]. The standard format used for the trailer is: Co-authored-by: Devin devin@cognition.ai While earlier or alternative configurations may have utilized other patterns, current documentation emphasizes the transition to official GitHub App bot integrations for improved attribution and compatibility [3]. Depending on the specific integration setup (such as the Devin GitHub App), the trailer may reference the official GitHub App bot identity (e.g., devin-ai-integration[bot]) to ensure clear provenance within repository histories [4]. Users are encouraged to check their specific repository's Devin integration documentation for the most accurate, environment-specific attribution string [5][3].
Citations:
- 1: https://arxiv.org/html/2603.28592v2
- 2: https://notes.subhohalder.com/p/i-stopped-calling-it-vibe-check
- 3: https://docs.devin.ai/cli/changelog/stable
- 4: https://github.com/powerset-co/github-coding-agent-tracker
- 5: https://docs.devin.ai/integrations/gh
Restrict DEVIN to a bot identity
Devin\s*(?:<[^>]*>)? still matches any human co-author named Devin with any email, so Co-authored-by: Devin <devin@company.com> is classified as AI and rejected in forbid mode. Anchor this to Devin’s bot email/identity instead of a bare name.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@commit_check/ai_signatures_data.py` around lines 159 - 169, The DEVIN
signature is too broad because the current Co-authored-by matcher in
ai_signatures_data.py accepts any person named Devin, so tighten the KnownAiTool
pattern to match Devin’s specific bot identity/email instead of a bare name.
Update the DEVIN entry and its _trailer matcher so only the intended bot
co-author format is recognized, and ensure human co-authors named Devin no
longer match.
1 participant
Summary
Add AI attribution governance to commit-check, enabling projects to forbid commits that contain known AI tool signatures at the commit-message level. This directly addresses the ongoing industry-wide discussion around AI disclosure in open source.
Background
The open-source ecosystem is converging on the need for AI attribution policies, but no tool exists at the CI level to help projects enforce them. This PR fills that gap:
Assisted-by:trailer formatCo-authored-bywithAssisted-byfor AI agentscommit-check is uniquely positioned to be the neutral enforcement layer.
Configuration
A single binary switch:
"ignore"-- no validation (default, backward compatible)"forbid"-- rejects any commit with known AI tool signatures, regardless of trailer style (Co-authored-by,Assisted-by, body markers, etc.)No
requiremode, noai_trailer_style-- the signature database recognizes all known formats automatically.Key Components
commit_check/ai_signatures.py-- Curated AI tool signature databaseDetects known AI tool markers in commit messages, including:
Co-authored-by: Claude,Assisted-by: Claude:<model> [tools],🤖 Generated with Claude,Claude-Session:,Claude-Workflow:Co-authored-by: CopilotCo-authored-by: CodexCo-authored-by: GeminiCo-authored-by: CursorCo-authored-by: DevinCo-authored-by: Aider,Co-authored-by: ... (aider)Co-authored-by: WindsurfCo-authored-by: TabbyAssisted-by: <tool>:<model> [tools](kernel style), model names (claude-sonnet-4,gpt-4-turbo)Built-in false positive prevention
Co-authored-by: Jane Doe <jane@example.com>are never flaggedAssisted-by:format accepts optional trailing tool list (Claude:claude-3-opus coccinelle sparse)Full integration
--ai-attribution=forbid[commit] ai_attribution = "forbid"CCHK_AI_ATTRIBUTION=forbidvalidate_message()includes AI attribution checks--format json: AI check results included in structured outputTest coverage
414 tests pass with zero regressions:
forbid/ignorepolicy coverageDocumentation updated
README.rst-- overview mentions AI attribution, example config includesai_attributiondocs/configuration.rst-- option table, env var mapping, CLI mapping, and example configdocs/what-is-new.rst-- full v2.10.0 changelog entry with detected tools listFuture work (not in this PR)
describe_validation_rulesto auto-comply--format json: Richer AI signature metadata for SBOM/audit toolingSummary by CodeRabbit
New Features
Bug Fixes
Documentation