fix(ci): decouple dashboard/report smoke from stand-only sts_audit by PhysShell · Pull Request #32 · PhysShell/OwnAudit

PhysShell · 2026-07-02T14:42:52Z

What & why

The SARIF export + dashboard smoke CI job is red on main (regression that merged with #31). #31 flipped build_dashboard.py's default DATA_DIR from sts_audit to the gitignored, CI-empty artifacts/, and the smoke invokes it with no argument:

python3 viz/build_dashboard.py
FileNotFoundError: .../artifacts/findings.json

Both smoke steps also read the stand-only, proprietary sts_audit/ corpus — which is slated for removal — so pointing CI back at it would only defer the break (and keeps real project data in the repo). Instead this decouples CI from sts_audit entirely, using a committed synthetic golden fixture.

Changes

file	change
`viz/fixtures/findings.json`, `viz/fixtures/health-report.md`	new — a small, fully synthetic golden set (invented module/file/handler names). Exercises every tool (`own-check`/`roslyn`/`codeql`/`infersharp`), all four fix tiers (T1–T4), every source family, multiple categories, and all four OWN leak-shapes, so the smoke still catches Python/JS-template regressions.
`.github/workflows/ci.yml`	point both steps at the fixture: `report.cli --findings viz/fixtures/findings.json` and `build_dashboard.py viz/fixtures`.
`viz/build_dashboard.py`	skip cleanly (exit 0, message) when `findings.json`/`health-report.md` are absent — an empty default `artifacts/` is "no work", not a hard failure.
`.gitignore`	ignore the generated `viz/arch-dashboard.html` (sibling of the already-ignored `sts-dashboard.html`).

Once sts_audit/ is deleted, nothing in CI breaks: neither step references it any more.

How verified

Locally, reproducing the CI job:

report.cli --min-level warning --findings viz/fixtures/findings.json → 14 findings → 11 SARIF results, report/out/ written.
build_dashboard.py viz/fixtures → renders (14 findings, 4 modules).
build_arch_dashboard.py → empty-state (unchanged).
build_dashboard.py artifacts (empty) and a nonexistent dir → both skip, exit 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DFSpD9yXgXePrFFX62XZ53

Generated by Claude Code

Summary by CodeRabbit

New Features
- Added committed sample data for dashboard and report smoke checks, improving confidence in generated output.
- CI now uses the bundled sample data for dashboard/report validation.
Bug Fixes
- Dashboard generation now handles missing input more gracefully and skips output when data isn’t available.
- Generated dashboard HTML is now excluded from version control.

The dashboard smoke ('SARIF export + dashboard smoke') failed after the DATA_DIR default flipped to the gitignored, CI-empty 'artifacts/' (FileNotFoundError on artifacts/findings.json). Both CI steps also read the stand-only, proprietary sts_audit corpus, which is slated for removal. - add viz/fixtures/{findings.json,health-report.md}: a small SYNTHETIC golden set (invented names) exercising every tool, tier, source family, category and OWN leak-shape, so the smoke keeps catching template regressions without depending on sts_audit or leaking real data. - point both CI steps at the fixture (report.cli --findings, build_dashboard positional DATA_DIR). - build_dashboard: skip cleanly (exit 0) when findings.json/health-report.md are absent, so an empty default artifacts/ is 'no work', not a hard failure. - gitignore the generated viz/arch-dashboard.html (sibling of sts-dashboard.html). Verified locally: report.cli + build_dashboard render from the fixture; empty and missing dirs skip with exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DFSpD9yXgXePrFFX62XZ53

coderabbitai · 2026-07-02T14:43:11Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: dbe5ac96-ae76-4d61-9d1c-fe9a95efa256

📥 Commits

Reviewing files that changed from the base of the PR and between f9b4af4 and 78132ca.

📒 Files selected for processing (5)

.github/workflows/ci.yml
.gitignore
viz/build_dashboard.py
viz/fixtures/findings.json
viz/fixtures/health-report.md

📝 Walkthrough

Walkthrough

Adds committed synthetic fixtures (findings.json, health-report.md) for CI smoke testing, updates viz/build_dashboard.py to make data_dir optional with skip-on-missing-file behavior versus error-on-explicit-missing, updates ci.yml to reference the fixtures directory, and ignores the generated dashboard HTML.

Changes

Fixture-driven CI dashboard smoke test

Layer / File(s)	Summary
Golden fixtures for findings and health report `viz/fixtures/findings.json`, `viz/fixtures/health-report.md`	New synthetic fixtures providing findings coverage metadata and a health report with module pain metrics for CI smoke testing.
Optional data_dir handling `viz/build_dashboard.py`	`main()` now defaults `data_dir` to `None`, skipping HTML generation silently when using the implicit `artifacts/` folder and required files are missing, or erroring via `ap.error` when an explicit directory lacks required files.
CI workflow and gitignore wiring `.github/workflows/ci.yml`, `.gitignore`	The `audit-report` job now passes `--findings viz/fixtures/findings.json` and runs `build_dashboard.py` against `viz/fixtures`; the generated `viz/arch-dashboard.html` is now gitignored.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CI as CI Workflow
  participant Report as report.cli
  participant Dashboard as build_dashboard.py
  participant Fixtures as viz/fixtures

  CI->>Report: run with --findings viz/fixtures/findings.json
  Report->>Fixtures: read findings.json
  CI->>Dashboard: run with viz/fixtures argument
  Dashboard->>Fixtures: read findings.json, health-report.md
  Dashboard-->>CI: generate dashboard HTML or skip if missing

Possibly related PRs

PhysShell/OwnAudit#7: Directly overlaps with the viz/build_dashboard.py input-handling changes for findings.json/health-report.md.
PhysShell/OwnAudit#9: Evolves the same CI workflow audit-report job SARIF/dashboard smoke-test setup.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the CI smoke-job change and its decoupling from the stand-only sts_audit corpus.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/ownaudit-pr-monitoring-4jniwg

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a9218373f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-02T14:46:32Z

+    if missing:
+        print(f"build_dashboard: {STS} has no {', '.join(missing)} — nothing to render, skipping.")
+        return


Make fixture smoke fail when inputs are missing

When the caller supplies the new fixture path, treating missing inputs as success makes the dashboard smoke a no-op: report.cli only requires findings.json, so if viz/fixtures/health-report.md is deleted/renamed or the CI path points at an empty directory, the workflow's python3 viz/build_dashboard.py viz/fixtures still exits 0 here without rendering the dashboard. Limit the clean skip to the default empty artifacts/ case, or make CI assert the fixture files exist before relying on the smoke.

Useful? React with 👍 / 👎.

Good catch — fixed in 78132ca (your first option). The clean skip now applies only to the implicit default artifacts/ (empty until a stand run populates it). An explicitly-supplied dir missing findings.json/health-report.md is now a hard error (argparse exit 2), so CI's build_dashboard.py viz/fixtures fails loudly if the fixture is ever deleted/renamed — no more silent no-op.

Verified: explicit valid → render (exit 0); default empty artifacts/ → skip (exit 0); explicit dir missing a file / nonexistent → exit 2.

Generated by Claude Code

… explicit Addresses Codex review on #32: the clean skip made the fixture smoke a no-op — an explicit `build_dashboard.py viz/fixtures` would still exit 0 if a fixture file were deleted/renamed, silently passing CI. Now the skip applies ONLY to the implicit default `artifacts/` (empty until a stand run); an explicitly-supplied dir missing findings.json/health-report.md is a hard error (exit 2), so CI fails loudly on a broken fixture. Verified: explicit valid -> render; default empty -> skip exit 0; explicit missing/nonexistent -> exit 2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DFSpD9yXgXePrFFX62XZ53

PhysShell · 2026-07-02T15:25:43Z

@coderabbitai review

Generated by Claude Code

coderabbitai · 2026-07-02T15:25:56Z

@PhysShell, sounds good — I'll review the changes now.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

chatgpt-codex-connector Bot reviewed Jul 2, 2026

View reviewed changes

PhysShell merged commit 581a437 into main Jul 2, 2026
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(ci): decouple dashboard/report smoke from stand-only sts_audit - #32

fix(ci): decouple dashboard/report smoke from stand-only sts_audit#32
PhysShell merged 2 commits into
mainfrom
claude/ownaudit-pr-monitoring-4jniwg

PhysShell commented Jul 2, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 2, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Jul 2, 2026

Uh oh!

PhysShell Jul 2, 2026

Uh oh!

PhysShell commented Jul 2, 2026

Uh oh!

coderabbitai Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

PhysShell commented Jul 2, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What & why

Changes

How verified

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

PhysShell Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

PhysShell commented Jul 2, 2026

Uh oh!

coderabbitai Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

PhysShell commented Jul 2, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 2, 2026 •

edited

Loading

coderabbitai Bot commented Jul 2, 2026 •

edited

Loading