fix(oracle): count Infer# PULSE_RESOURCE_LEAK + product-only scoping (--exclude-tests)

[PhysShell]

Make the first three-way run correct: Infer# leaks + product-only scope

The first real Own.NET / Infer# / CodeQL run on DapperLib/Dapper finally fired all three tools — and exposed two issues in the comparator. Both fixed here.

1. Infer# showed 0 leaks — it actually found 3

Modern Infer# (Pulse engine, v1.2+) emits PULSE_RESOURCE_LEAK, not the older RESOURCE_LEAK. The comparator's INFER_LEAK set predated that rename, so Infer#'s 3 resource leaks were mis-classified as "out of scope" context (you could see infersharp:PULSE_RESOURCE_LEAK x3 sitting in the 269-item context bucket).

→ Match the Infer leak family by substring (*RESOURCE_LEAK* / *MEMORY_LEAK*), robust to version drift. Real totals: Own.NET 1 / Infer# 3 / CodeQL 15.

2. The three tools were comparing different code

Infer# builds only the product project (the #24 heuristic), but own-check and CodeQL scan the whole source tree — so their leak counts were inflated by tests//benchmarks/ code Infer# never saw, and agree was depressed because the tools literally looked at different files.

→ New --exclude-tests (comparator) + an include_tests workflow input (default off ⇒ product-only). Findings under test/benchmark/sample/example paths are dropped uniformly across all tools, so the diff is apples-to-apples. Done in the comparator (not per-tool) because CodeQL's paths-ignore is unreliable for compiled C# with build-mode: none. The report header notes the scope; include_tests=true compares across everything.

What the Dapper recall gap actually was (for context)

Of CodeQL's 15: 6 are exception-path (cs/dispose-not-called-on-throw — we don't model exceptional edges, by design), 6 are DataTable (we treat it as dispose-optional — our Task/DataTable exemption that keeps precision), and 3 are genuine coverage gaps (SqlCommand / UnitOfWork / CancellationTokenSource — async methods we skip, or unresolved custom IDisposable). So our gap is policy + coverage, not logic — a strong case for the extractor's --stats and async support next.

Verified

oracle_compare.py --selftest 15/15 (adds PULSE classification + the exclude-tests predicate) · exclude-tests smoke check passes · oracle.yml valid YAML · ruff clean · run_tests.py exit 0. No change to the three tools — comparator + docs + one workflow input only.

After merge

Re-dispatch oracle (cross-tool), repo = DapperLib/Dapper, all inputs empty: Infer# now counts (3), and the diff is restricted to Dapper's product code — so agree finally has a chance to be non-zero.

🤖 Generated with Claude Code

---

Generated by Claude Code

Summary by CodeRabbit

• New Features

  • Added an include_tests workflow option to control whether test/benchmark/sample/example code is included in oracle comparisons (default: excluded).
  • Comparison output now clearly indicates “product code only” scope when non-product paths are excluded, including the number excluded.

• Bug Fixes

  • Improved oracle leak classification to better match leak-related rule IDs (including Pulse leak IDs).

• Documentation

  • Updated oracle comparison notes to reflect the refined default scoping behavior and how to enable test inclusion.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d6cf0d82-7818-401b-adfd-341a43311914

📥 Commits

Reviewing files that changed from the base of the PR and between 4c35043 and ae11bff.

📒 Files selected for processing (2)

• .github/workflows/oracle.yml
• scripts/oracle_compare.py

🚧 Files skipped from review as they are similar to previous changes (2)

• .github/workflows/oracle.yml
• scripts/oracle_compare.py

---

📝 Walkthrough

Walkthrough

The PR adds test-path exclusion to the oracle comparison pipeline. A new _is_test_path() predicate and --exclude-tests CLI flag are added to oracle_compare.py, along with render_md() parameters to surface the excluded count. Infer# leak rule classification is updated to substring matching for Pulse rule names. The GitHub Actions workflow gains an include_tests input that conditionally passes --exclude-tests to the script. Documentation is updated accordingly.

Changes

Oracle comparator: exclude-tests filtering + Pulse rule fix

Layer / File(s)	Summary
Pulse leak rule classification fix scripts/oracle_compare.py	INFER_LEAK rule set and _oracle_class() updated to use case-insensitive substring matching on RESOURCE_LEAK/MEMORY_LEAK instead of exact membership, covering Pulse-renamed rule IDs.
Test-path filtering feature scripts/oracle_compare.py	Adds _is_test_path() for segment-based detection of test/benchmark/sample/example paths, a --exclude-tests CLI flag, filtering in main() that counts excluded findings, and new excluded_tests and exclude_tests_mode parameters in render_md() that conditionally emit a "product code only" scope line.
Selftest coverage scripts/oracle_compare.py	_selftest() extended with assertions for PULSE_RESOURCE_LEAK classification and _is_test_path() correctness; selftest total updated.
Workflow input and conditional flag wiring .github/workflows/oracle.yml	Adds include_tests workflow_dispatch input (default "false") and wires INCLUDE_TESTS into the diff step, conditionally appending --exclude-tests to the comparator invocation.
Documentation updates docs/notes/oracle.md	Corrects the Infer# rule name in the comparison mapping, documents the default product-only comparison scope with excluded path patterns, and updates the example workflow snippet to reflect include_tests = false.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• PhysShell/Own.NET#21: Introduced the cross-tool oracle workflow and oracle_compare.py that this PR directly extends with test-path filtering and Pulse rule classification.

Poem

🐇 Hopping through the diff, I see,
No test paths in the oracle spree!
PULSE_RESOURCE_LEAK? Now matched with care,
--exclude-tests floats through the air.
Product code only, crisp and clean —
The sharpest oracle diff I've seen! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the two main changes: fixing Infer# PULSE_RESOURCE_LEAK classification and adding product-only scoping with --exclude-tests flag.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/zen-pasteur-76hfs1

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/oracle.yml:
- Around line 37-40: The `include_tests` input is defined as a free-form string
without type specification, causing case-sensitive comparisons to fail for
values like "True" or "TRUE". Add `type: boolean` to the `include_tests` input
definition to normalize it to a string literal "true" or "false", and update the
condition at line 172 where this input is evaluated to use the
`${INCLUDE_TESTS,,}` parameter expansion operator for case-insensitive matching
instead of exact lowercase comparison.

In `@scripts/oracle_compare.py`:
- Around line 239-241: The scope note is currently shown only when
excluded_tests has a value greater than zero, which creates ambiguity when the
exclude-tests mode is enabled but no tests are actually excluded. Instead of
checking if excluded_tests is non-zero in the conditional at lines 239-241,
check if the exclude-tests flag or mode is enabled (the configuration setting
that controls whether test filtering is active). This way the scope note will
always be displayed whenever product-only filtering is active, regardless of the
count. Apply the same fix to the similar conditional block at lines 426-427.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 38efc7e0-41dd-481c-a21a-3b01d74e93e6

📥 Commits

Reviewing files that changed from the base of the PR and between a9239e1 and 4c35043.

📒 Files selected for processing (3)

• .github/workflows/oracle.yml
• docs/notes/oracle.md
• scripts/oracle_compare.py