Skip to content

Add merged-SARIF and HTML renderers — complete Phase 1 audit outputs#101

Merged
PhysShell merged 3 commits into
mainfrom
claude/brave-gauss-elvquy
Jun 25, 2026
Merged

Add merged-SARIF and HTML renderers — complete Phase 1 audit outputs#101
PhysShell merged 3 commits into
mainfrom
claude/brave-gauss-elvquy

Conversation

@PhysShell

@PhysShell PhysShell commented Jun 25, 2026

Copy link
Copy Markdown
Owner

Completes the Phase 1 static deliverable (Plan.md §3.5) by adding the two
remaining report renderers over the same scored model — no new analysis, just two
more views.

What's here

  • merged SARIF (render_sarif) — one SARIF 2.1.0 log where the audit
    categories are the rules (own-audit/<category>), so GitHub's Security tab
    groups by our taxonomy rather than the underlying tool ids. Each scored cluster
    is a result carrying its contributing tools, confidence, severity and pain in
    properties; severities map P0/P1→error, P2→warning, P3→note. The
    suppressed / analysis-skipped counts ride in the run properties so the honesty
    ledger travels with the log. Upload it to GitHub code scanning.
  • HTML (render_html) — a self-contained heatmap page (no external assets):
    the pain-ranked module table, high-confidence/candidate findings, and the
    coverage section. All interpolated text is html-escaped.

run_static.aggregate now writes report.sarif and report.html next to
report.md and report.json.

Tests

Extended the report and run_static selftests to cover both renderers (SARIF
result/rule/level mapping + run properties; HTML structure + coverage section).

normalize 19/19 · score 11/11 · report 31/31 · run_static 14/14 · ruff clean

With this, Phase 1 (static, build-free) is complete: injection mechanism,
build-free runners, normalization + taxonomy, agreement scoring, the pain heatmap,
and all four renderers, gated by the audit-selftests CI job.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE


Generated by Claude Code

Summary by CodeRabbit

  • New Features
    • Added two new report formats: merged SARIF (2.1.0) and a self-contained HTML heatmap.
    • The static audit workflow now generates Markdown, JSON, SARIF, and HTML outputs.
    • Report generation CLI now supports sarif and html formats.
  • Bug Fixes
    • Enhanced report coverage rendering to include suppressed and “analysis skipped” counts/notes.
  • Tests
    • Expanded end-to-end validation to confirm SARIF and HTML artifacts are written and contain expected markers.

…outputs)

The static health report now renders in all four formats over the same scored
model:

- merged SARIF (render_sarif): one SARIF 2.1.0 log where audit categories are the
  rules (own-audit/<category>), each scored cluster is a result carrying its tools,
  confidence, severity and pain in properties, and P0/P1->error, P2->warning,
  P3->note. Suppressed/analysis-skipped counts ride in the run properties so the
  honesty ledger travels with the log. Upload to GitHub code scanning.
- HTML (render_html): a self-contained heatmap page (no external assets) with the
  pain-ranked module table, high-confidence/candidate findings, and the coverage
  section; all interpolated text is html-escaped.

run_static.aggregate now writes report.sarif and report.html alongside report.md
and report.json. Extended the report and run_static selftests to cover both new
renderers (SARIF result/rule/level mapping + run props; HTML structure + coverage);
report selftest 31/31, run_static 14/14. Updated the README status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 36d7e2fc-98f9-4199-998b-92b113aa50a3

📥 Commits

Reviewing files that changed from the base of the PR and between 70ef8a0 and 6482151.

📒 Files selected for processing (2)
  • audit/aggregate/report.py
  • audit/static/run_static.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • audit/static/run_static.py
  • audit/aggregate/report.py

📝 Walkthrough

Walkthrough

Added SARIF and HTML report renderers to the audit aggregation flow, extended the static runner to write and verify those artifacts, and updated the README to describe the full report set.

Changes

Audit report exports

Layer / File(s) Summary
Report contract and coverage updates
audit/aggregate/report.py
Declares the new report formats, maps audit severities to SARIF levels, and extends coverage output with analysis-skipped details.
SARIF and HTML renderers
audit/aggregate/report.py
Builds merged SARIF output and a self-contained HTML report, adds formatter dispatch for the new formats, and expands self-tests for SARIF and HTML output.
Static runner and README updates
audit/static/run_static.py, audit/README.md
Writes report.sarif and report.html, checks them in self-tests, and updates the README status text.

Sequence Diagram(s)

sequenceDiagram
  participant main as main(...)
  participant report as audit/aggregate/report.py
  participant static as audit/static/run_static.py
  participant selftest as _selftest()

  main->>report: --format sarif / --format html
  static->>report: render_sarif(...)
  static->>report: render_html(...)
  selftest->>static: aggregate()
  selftest->>static: verify report.sarif and report.html
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

A bunny found four reports in a row,
With SARIF and HTML all aglow.
I twitched my nose, gave a happy hop—
And tucked the audit trail neatly on top. 🐇

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main change: adding merged SARIF and HTML renderers to complete the Phase 1 audit outputs.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/brave-gauss-elvquy

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ae659b8b54

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread audit/aggregate/report.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
audit/static/run_static.py (1)

76-83: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Refresh the aggregate() contract text.

These lines now write and return SARIF/HTML artifacts too, but the function docstring still says it writes only markdown + json and “returns the scored totals.” Please update that contract so callers can discover the new outputs from the definition itself.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@audit/static/run_static.py` around lines 76 - 83, Update the aggregate()
contract text in run_static.py to match its current behavior: it now writes
SARIF and HTML artifacts in addition to markdown and JSON, and returns a richer
result than just scored totals. Refresh the docstring near aggregate() so it
explicitly lists all generated outputs and the returned keys, including
report_sarif and report_html, so callers can rely on the function definition for
the full contract.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@audit/aggregate/report.py`:
- Around line 192-197: The HTML coverage ledger in _coverage_section() is
missing the analysis_skipped bucket even though render_sarif() already carries
it. Update the coverage section rendering to surface analysis_skipped alongside
the existing totals, kept, and suppressed values, using the same coverage data
flow through report.py. Also extend _selftest() with a note=True
analysis-skipped fixture so the HTML report and SARIF output stay aligned and
regressions are caught.

---

Nitpick comments:
In `@audit/static/run_static.py`:
- Around line 76-83: Update the aggregate() contract text in run_static.py to
match its current behavior: it now writes SARIF and HTML artifacts in addition
to markdown and JSON, and returns a richer result than just scored totals.
Refresh the docstring near aggregate() so it explicitly lists all generated
outputs and the returned keys, including report_sarif and report_html, so
callers can rely on the function definition for the full contract.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: bef226aa-68f5-4376-b3a6-72a6ec4064db

📥 Commits

Reviewing files that changed from the base of the PR and between 6904a0a and ae659b8.

📒 Files selected for processing (3)
  • audit/README.md
  • audit/aggregate/report.py
  • audit/static/run_static.py

Comment thread audit/aggregate/report.py
claude added 2 commits June 25, 2026 04:57
render_sarif fabricated a line-1 region for findings parse_sarif recorded as
line 0 (an upstream result with no region.startLine — a file-level diagnostic).
On code-scanning upload that mis-pinned those alerts to the first line. Now the
region is emitted only when the line is >= 1; otherwise the physicalLocation
stays file-level (artifactLocation only), preserving the upstream intent. Added a
selftest asserting a line-0 finding omits the region.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
…) contract

CodeRabbit (PR #101):
- _coverage_section() now renders the analysis_skipped bucket (own-check OWN050
  coverage notes) alongside total/kept/suppressed. It was counted in coverage()
  and carried in the SARIF run properties since #100, but the human markdown/HTML
  report never showed it — so notes silently vanished from the honesty ledger.
  Added a note=True fixture to the report selftest and assertions that both the
  markdown and HTML coverage sections surface it, keeping SARIF/HTML aligned.
- Refreshed the run_static.aggregate() docstring to state it writes all four
  artifacts (report.md/json/sarif/html) and returns the four report_* paths.

report selftest 34/34, run_static 14/14, ruff clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
@PhysShell PhysShell merged commit f67630c into main Jun 25, 2026
24 checks passed
PhysShell pushed a commit that referenced this pull request Jun 25, 2026
…lan.md §4)

The runtime layer confirms static findings by observing the running app, and its
findings flow through the SAME normalize -> score -> report pipeline as the static
tiers — so a runtime-confirmed leak in the same file as a static finding clusters
with it into one high-confidence result (the static->runtime confirmation in
Plan.md §3.5).

Testable core (pure Python, Linux-CI-gated):
- audit/runtime/ingest.py converts the leak-harness JSON result into SARIF; only
  findings whose deterministic growth assertion tripped (leaked:true) become
  results, type-based findings stay file-level (reusing the #101 region fix). Its
  --selftest proves runtime rules categorize correctly AND that a static OWN014 +
  a runtime leak in the same file form one high-confidence {own-check, leak-harness}
  cluster.
- categories.yml gains RUNTIME-* rule mappings for categories 2/3/4/11 (4 and 11
  were NO-TOOL for static — the runtime layer is now their only tool) plus their
  severity baselines.
- CI's audit-selftests job runs the ingest selftest.

Windows / build-required skeleton (NOT CI-gated, per the build-required tier):
- audit/runtime/LeakHarness/ — net472 C# harness (FlaUI + procdump + ClrMD): the
  deterministic GC+snapshot loop, growth assertion, and JSON result. Program.cs /
  Scenario.cs / HeapCounter.cs / LeakHarness.csproj.
- audit/runtime/scenarios/open-close-declaration.yml — one declarative scenario
  with its schema documented inline.
- audit/runtime/README.md documents the stack, flow, and how it plugs into the
  unified report.

All five audit selftests green (normalize/score/report/run_static/ingest), ruff
clean tree-wide.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QDPpNT9Uh8RoTrKPvgcoRE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants