oracle: exclude doc/snippet trees + field-notes pattern log (repo requirement)#110
Merged
Conversation
Dispatch via API is blocked for the automation token (403), so use the documented push fallback: run the cross-tool oracle by bumping the sentinel corpus/oracle-target.txt on the active dev branch. Repoint the push trigger from the stale branch to claude/agenda-2rufsj and add a re-run note so the three-way diff (Own.NET vs Infer# vs CodeQL) runs on the systemevents-console fixture — the Linux-buildable target where all three tools, Infer# included, fire. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
Re-run the documented Dapper worked example on a live clone after the P-014 Tier A/B work to confirm the full cross-tool pipeline (clone -> build -> own-check + CodeQL + Infer#) still holds on external code. Expected baseline: agree 0, own-only 0 (no false positives), oracle-only ~5 (Infer# ownership-transfer over-reports + CodeQL dispose-on-throw). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
Exploratory cross-tool run on a fresh external target with no documented baseline, to see if Own.NET surfaces own-only (candidate FPs) or oracle-only (recall gaps) on code it was not tuned against. Polly v8 targets net8/net6/netstandard2.0 only, so it builds on the dotnet-8 runner and all three tools should fire. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
…rked example Two findings from the blind Polly oracle run (own-only 0, oracle-only 38): 1. _is_test_path now treats doc/docs/snippet* path segments as non-product, alongside test/benchmark/sample/example. Polly's src/Snippets/Docs/* is illustrative code that intentionally never disposes ~20 HttpResponseMessage/HttpClient/rate-limiter examples; counting them as product leaks inflated the oracle-only recall gap with code that was never meant to dispose. Selftest extended to cover the doc/snippet exclusion; 24/24 green. 2. docs/notes/oracle.md gains a second worked example (Polly), recording the ownership-transfer precision win: BulkheadSemaphoreFactory returns two SemaphoreSlim that BulkheadPolicy owns and disposes in Dispose() — CodeQL's cs/local-not-disposed flags the factory (a false positive it can't trace through tuple-return ownership), and Own.NET is correctly silent. Second live ownership-transfer win after Dapper's DbWrappedReader. The note decomposes the full oracle-only 38 (Infer# struct-using over-reports + doc snippets + 2 CodeQL FPs + 1 by-design exceptional-path skip) so the headline number is read honestly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
Bump the sentinel to re-trigger the oracle on Polly after fd6294e. The ~20 src/Snippets/Docs/* CodeQL findings should now drop as non-product: oracle-only 38 -> ~18, own-only still 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
…runs A running notebook of real resource/lifetime idioms spotted while running the cross-tool oracle at open-source repos (Polly, Dapper): ownership transfer via factory return, deferred disposal via lifecycle callback, pooled rent/return disposables, struct-using scoped locks, the two-semaphore bulkhead, and dispose-forwarding wrappers. Each entry pins the source file and the analyzer angle (why naive leak detectors over-report and a transfer/escape-aware checker correctly stays silent) — so the collection doubles as a learning resource and a map of Own.NET's precision frontier. Cross-linked from oracle.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
Codify the pattern-logging convention normatively in three places so it binds future runs (and contributors/agents), not just lives as a soft note at the bottom of the notebook: - README.md: a 'Требование (convention)' callout next to the oracle description — every oracle run obliges triaging oracle-only and recording any FP/by-design idiom in field-notes-patterns.md. - oracle.md: a '## Maintenance requirement (not optional)' section in the run procedure — a run that surfaces a new FP/by-design idiom and doesn't append it is an incomplete run. - field-notes-patterns.md: the closing note rewritten as a normative '## Maintaining this notebook (a repo requirement)', with the source-pinned + honest-confidence discipline spelled out. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
Revert .github/workflows/oracle.yml and corpus/oracle-target.txt to main: the push-trigger repoint and target bumps were dev-loop scaffolding used to run the oracle (the automation token can't workflow_dispatch), not changes meant for main — the workflow's own comment says the push-trigger block is dev-branch-only. The PR keeps just the substantive work: the _is_test_path doc/snippet exclusion fix and the field-notes / oracle.md / README documentation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughThis PR broadens oracle path filtering and adds documentation for triaging ChangesOracle triage maintenance
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b5bc1c592
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for seg in path.lower().split("/"): | ||
| if seg in ("test", "tests") or seg.startswith(("benchmark", "sample", "example")): | ||
| if (seg in ("test", "tests", "doc", "docs") | ||
| or seg.startswith(("benchmark", "sample", "example", "snippet"))): |
There was a problem hiding this comment.
Because the oracle workflow adds --exclude-tests by default unless include_tests is true, this predicate now removes every finding under any product directory whose segment merely starts with snippet (for example src/SnippetEngine/Foo.cs or src/SnippetService/...). Polly's src/Snippets/Docs/* case is already covered by the exact docs segment, so the new broad prefix can silently suppress real product leaks from the report and JSON totals; restrict this to exact snippet/snippets or require an accompanying docs segment.
Useful? React with 👍 / 👎.
startswith('snippet') would drop real product dirs like SnippetEngine/
SnippetService. Make snippet/snippets exact-match (Polly's
src/Snippets/Docs/* stays covered by the exact snippets+docs segments);
keep prefix match only for the long unambiguous benchmark/sample/example.
Selftest now asserts src/SnippetEngine and src/Documentation are NOT
treated as non-product.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/notes/oracle.md`:
- Around line 157-208: The Polly bucket totals in the worked example do not
reconcile cleanly with the stated oracle-only total, so update the summary in
the documentation to make the counts consistent. Check the breakdown in the
Polly section around the Infer#, CodeQL, and genuinely-product findings, then
either correct the individual bucket counts so they sum to the stated
oracle-only 38 or explicitly mark the per-bucket numbers as approximate. Keep
the wording aligned with the existing Polly narrative and the cited symbols like
TimedLock, BulkheadSemaphoreFactory, ConfigureBuilderContextExtensions, and
TimeoutResilienceStrategy.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 19f20ce6-dfa6-46f2-bd82-bdfbc04e2a85
📒 Files selected for processing (4)
README.mddocs/notes/field-notes-patterns.mddocs/notes/oracle.mdscripts/oracle_compare.py
The sub-buckets now sum exactly to oracle-only 38 = 12 Infer# TimedLock over-reports + 22 CodeQL src/Snippets/Docs findings + 4 product CodeQL (3 FPs across BulkheadSemaphoreFactory:8/:11 + ConfigureBuilderContext:40, plus 1 by-design TimeoutResilienceStrategy:67). Were '~20' + 'a handful'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fmwk5ruExKy5fQ77CJ2NtF
This was referenced Jun 27, 2026
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two things, both born from running the cross-tool oracle against real OSS repos
(Dapper, then a blind run on App-vNext/Polly):
1. Harness fix — exclude doc/snippet trees from the product diff
scripts/oracle_compare.py_is_test_pathnow treatsdoc/docs/snippet*path segments as non-product, alongside
test/benchmark/sample/example.Polly's
src/Snippets/Docs/*is illustrative documentation code thatintentionally never disposes ~20
HttpResponseMessage/HttpClient/rate-limiterexamples. Counting those as product leaks inflated the
oracle-onlyrecall gapwith code that was never meant to dispose. Confirmed live on Polly:
The remaining 16
oracle-onlyis now a fully-explained set — 12 Infer#TimedLockstruct-usingover-reports + 3 triaged product findings (2 CodeQL FPson ownership-transfer / deferred-dispose + 1 by-design exceptional-path) — i.e.
zero genuine recall gap, precision held (own-only 0) on a third repo. Selftest
extended to cover the doc/snippet exclusion; 24/24 green.
2. Field-notes pattern log + a repo requirement to maintain it
New
docs/notes/field-notes-patterns.md: a curated, source-pinned collection ofreal C# resource/lifetime idioms found in the wild — ownership transfer via
factory return, deferred disposal via lifecycle callback, pooled rent/return
disposables, struct-
usingscoped locks, the two-semaphore bulkhead,dispose-forwarding wrappers. Each entry pins the source file and the analyzer
angle (why naive leak detectors over-report and why a transfer/escape-aware
checker correctly stays silent), so the doc doubles as a C# learning resource and
a map of Own.NET's precision frontier.
Codified as a standing convention (normative, in three cross-linked places —
README callout,
oracle.md§ Maintenance requirement, the notebook's own header):every oracle run must triage
oracle-onlyand record any FP/by-design idiom inthe notebook. A run that surfaces a new idiom and doesn't append it is an
incomplete run.
oracle.mdalso gains a second worked example (Polly) decomposingthe
oracle-only 38honestly.Files
scripts/oracle_compare.py—_is_test_pathwidened + selftest (24/24).docs/notes/field-notes-patterns.md— new pattern notebook.docs/notes/oracle.md— Polly worked example + Maintenance-requirement section + field-notes cross-link.README.md— convention callout next to the oracle description.Not included
The dev-loop scaffolding used to run the oracle (push-trigger repoint + target
sentinel bumps — the automation token can't
workflow_dispatch) was reverted tomain; it isn't meant to land. Pure additive value only.🤖 Generated with Claude Code
Generated by Claude Code
Summary by CodeRabbit
doc/docsandsnippet/snippetspaths as non-product, without mistakenly excluding similarly named product directories.