Add merged-SARIF and HTML renderers — complete Phase 1 audit outputs#101
Conversation
…outputs) The static health report now renders in all four formats over the same scored model: - merged SARIF (render_sarif): one SARIF 2.1.0 log where audit categories are the rules (own-audit/<category>), each scored cluster is a result carrying its tools, confidence, severity and pain in properties, and P0/P1->error, P2->warning, P3->note. Suppressed/analysis-skipped counts ride in the run properties so the honesty ledger travels with the log. Upload to GitHub code scanning. - HTML (render_html): a self-contained heatmap page (no external assets) with the pain-ranked module table, high-confidence/candidate findings, and the coverage section; all interpolated text is html-escaped. run_static.aggregate now writes report.sarif and report.html alongside report.md and report.json. Extended the report and run_static selftests to cover both new renderers (SARIF result/rule/level mapping + run props; HTML structure + coverage); report selftest 31/31, run_static 14/14. Updated the README status. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdded SARIF and HTML report renderers to the audit aggregation flow, extended the static runner to write and verify those artifacts, and updated the README to describe the full report set. ChangesAudit report exports
Sequence Diagram(s)sequenceDiagram
participant main as main(...)
participant report as audit/aggregate/report.py
participant static as audit/static/run_static.py
participant selftest as _selftest()
main->>report: --format sarif / --format html
static->>report: render_sarif(...)
static->>report: render_html(...)
selftest->>static: aggregate()
selftest->>static: verify report.sarif and report.html
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ae659b8b54
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
audit/static/run_static.py (1)
76-83: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRefresh the
aggregate()contract text.These lines now write and return SARIF/HTML artifacts too, but the function docstring still says it writes only markdown + json and “returns the scored totals.” Please update that contract so callers can discover the new outputs from the definition itself.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@audit/static/run_static.py` around lines 76 - 83, Update the aggregate() contract text in run_static.py to match its current behavior: it now writes SARIF and HTML artifacts in addition to markdown and JSON, and returns a richer result than just scored totals. Refresh the docstring near aggregate() so it explicitly lists all generated outputs and the returned keys, including report_sarif and report_html, so callers can rely on the function definition for the full contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@audit/aggregate/report.py`:
- Around line 192-197: The HTML coverage ledger in _coverage_section() is
missing the analysis_skipped bucket even though render_sarif() already carries
it. Update the coverage section rendering to surface analysis_skipped alongside
the existing totals, kept, and suppressed values, using the same coverage data
flow through report.py. Also extend _selftest() with a note=True
analysis-skipped fixture so the HTML report and SARIF output stay aligned and
regressions are caught.
---
Nitpick comments:
In `@audit/static/run_static.py`:
- Around line 76-83: Update the aggregate() contract text in run_static.py to
match its current behavior: it now writes SARIF and HTML artifacts in addition
to markdown and JSON, and returns a richer result than just scored totals.
Refresh the docstring near aggregate() so it explicitly lists all generated
outputs and the returned keys, including report_sarif and report_html, so
callers can rely on the function definition for the full contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: bef226aa-68f5-4376-b3a6-72a6ec4064db
📒 Files selected for processing (3)
audit/README.mdaudit/aggregate/report.pyaudit/static/run_static.py
render_sarif fabricated a line-1 region for findings parse_sarif recorded as line 0 (an upstream result with no region.startLine — a file-level diagnostic). On code-scanning upload that mis-pinned those alerts to the first line. Now the region is emitted only when the line is >= 1; otherwise the physicalLocation stays file-level (artifactLocation only), preserving the upstream intent. Added a selftest asserting a line-0 finding omits the region. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
…) contract CodeRabbit (PR #101): - _coverage_section() now renders the analysis_skipped bucket (own-check OWN050 coverage notes) alongside total/kept/suppressed. It was counted in coverage() and carried in the SARIF run properties since #100, but the human markdown/HTML report never showed it — so notes silently vanished from the honesty ledger. Added a note=True fixture to the report selftest and assertions that both the markdown and HTML coverage sections surface it, keeping SARIF/HTML aligned. - Refreshed the run_static.aggregate() docstring to state it writes all four artifacts (report.md/json/sarif/html) and returns the four report_* paths. report selftest 34/34, run_static 14/14, ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
…lan.md §4) The runtime layer confirms static findings by observing the running app, and its findings flow through the SAME normalize -> score -> report pipeline as the static tiers — so a runtime-confirmed leak in the same file as a static finding clusters with it into one high-confidence result (the static->runtime confirmation in Plan.md §3.5). Testable core (pure Python, Linux-CI-gated): - audit/runtime/ingest.py converts the leak-harness JSON result into SARIF; only findings whose deterministic growth assertion tripped (leaked:true) become results, type-based findings stay file-level (reusing the #101 region fix). Its --selftest proves runtime rules categorize correctly AND that a static OWN014 + a runtime leak in the same file form one high-confidence {own-check, leak-harness} cluster. - categories.yml gains RUNTIME-* rule mappings for categories 2/3/4/11 (4 and 11 were NO-TOOL for static — the runtime layer is now their only tool) plus their severity baselines. - CI's audit-selftests job runs the ingest selftest. Windows / build-required skeleton (NOT CI-gated, per the build-required tier): - audit/runtime/LeakHarness/ — net472 C# harness (FlaUI + procdump + ClrMD): the deterministic GC+snapshot loop, growth assertion, and JSON result. Program.cs / Scenario.cs / HeapCounter.cs / LeakHarness.csproj. - audit/runtime/scenarios/open-close-declaration.yml — one declarative scenario with its schema documented inline. - audit/runtime/README.md documents the stack, flow, and how it plugs into the unified report. All five audit selftests green (normalize/score/report/run_static/ingest), ruff clean tree-wide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
Completes the Phase 1 static deliverable (Plan.md §3.5) by adding the two
remaining report renderers over the same scored model — no new analysis, just two
more views.
What's here
render_sarif) — one SARIF 2.1.0 log where the auditcategories are the rules (
own-audit/<category>), so GitHub's Security tabgroups by our taxonomy rather than the underlying tool ids. Each scored cluster
is a result carrying its contributing tools, confidence, severity and pain in
properties; severities map P0/P1→
error, P2→warning, P3→note. Thesuppressed / analysis-skipped counts ride in the run properties so the honesty
ledger travels with the log. Upload it to GitHub code scanning.
render_html) — a self-contained heatmap page (no external assets):the pain-ranked module table, high-confidence/candidate findings, and the
coverage section. All interpolated text is
html-escaped.run_static.aggregatenow writesreport.sarifandreport.htmlnext toreport.mdandreport.json.Tests
Extended the
reportandrun_staticselftests to cover both renderers (SARIFresult/rule/level mapping + run properties; HTML structure + coverage section).
With this, Phase 1 (static, build-free) is complete: injection mechanism,
build-free runners, normalization + taxonomy, agreement scoring, the pain heatmap,
and all four renderers, gated by the
audit-selftestsCI job.🤖 Generated with Claude Code
https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
Generated by Claude Code
Summary by CodeRabbit
sarifandhtmlformats.