Skip to content

Audit taxonomy: introduce rule-level risk lanes and separate CodeStyle from actionable findings #267

Description

@PhysShell

Classification

  • Change type: enhancement / reporting semantics
  • Priority: P0
  • Expected value: very high — prevents cosmetic findings from competing with correctness and lifetime risks
  • Implementation risk: medium — classification drift can silently alter dashboards
  • Canonical owner: Own.NET/audit/static/taxonomy/ and aggregation

Problem

The current taxonomy correctly keeps severity separate from cross-tool agreement, but category 14 (general-quality) is too broad. It combines correctness, performance, maintainability and style-oriented rule families under one P2 bucket.

That makes ranking and triage structurally dishonest: an auto-fixable formatting/style warning can compete with an async correctness problem or resource-lifetime finding merely because they share a wildcard family.

This must be fixed before any learned ranker or probability model is introduced. A model trained on a bad taxonomy will produce mathematically polished nonsense.

Goal

Introduce a rule-level registry with explicit risk lanes and presentation policy.

Initial lanes:

Security
Correctness
Reliability
LifetimeResource
Performance
Architecture
Maintainability
CodeStyle
CoverageUnknown

The exact public names may be adjusted in review, but CodeStyle and CoverageUnknown must remain distinct from actionable risk lanes.

Rule registry contract

Each characterized rule should support fields equivalent to:

RULE_ID:
  lane: Correctness
  impact: high | medium | low | none
  auto_fixable: true | false | unknown
  participates_in_risk_ranking: true | false
  default_presentation: triage | suggestion | fixer | hidden-legacy | coverage
  source_family: <independence/corroboration family>
  rationale: <short text>

Requirements:

  • exact rule mapping wins over family/glob fallback;
  • unknown rules remain visible in CoverageUnknown or uncategorized, never dropped;
  • wildcard mappings are conservative and cannot classify a mixed family as CodeStyle merely by prefix;
  • source family is recorded so two correlated frontends are not treated as independent corroboration later.

Initial characterization scope

Do not attempt to classify every analyzer rule in one heroic PR. Start with the rules that dominate current volume and product value:

  • OWN*;
  • XAML*;
  • RUNTIME-*;
  • CA2000, CA2213 and other explicitly used IDisposable rules;
  • top-N Meziantou rules by observed volume;
  • top-N Roslynator rules by observed volume;
  • top-N Sonar rules by observed volume;
  • CodeQL rules currently contributing to audit agreement.

The PR must publish the observed top-N inventory used to choose the first slice.

CodeStyle policy

Default behavior for CodeStyle:

  • does not participate in risk/pain ranking;
  • does not block merge by default;
  • auto-fixable rules route to the fixer lane;
  • findings in changed code may be shown as suggestions;
  • untouched legacy findings remain available in a debt report but do not flood PR triage;
  • promotion to blocking requires an explicit project policy, not an accidental severity inheritance.

Corroboration terminology

Replace or prepare to replace the current binary high/candidate confidence wording with evidence-oriented terminology:

independent
correlated
single_source

This issue does not implement calibrated probabilities. It only prevents cross-tool agreement from being mislabeled as statistical confidence.

Required controls

  1. A style-only rule and a correctness rule from the same analyzer family land in different lanes.
  2. An unknown rule is surfaced in coverage and contributes no phantom risk score.
  3. OWN001 resource splits remain intact.
  4. XAML/runtime findings preserve their existing category semantics while gaining a lane.
  5. A CodeStyle finding cannot outrank a P1 lifetime finding through the normal risk queue.
  6. Exact mappings override broad wildcards deterministically.
  7. Existing suppression counts remain counted and visible.

Acceptance

  • A versioned rule registry/lane contract exists.
  • Category 14 is no longer the sole semantic home for mixed correctness/performance/style findings.
  • Top-volume rules are characterized individually.
  • CodeStyle has an explicit non-blocking/fixer-oriented default policy.
  • Unknown rules remain visible as coverage, never silently classified or dropped.
  • Severity, lane, corroboration and presentation policy remain separate axes.
  • Current audit fixtures are updated with an explained semantic diff.

Non-goals

  • No learned ranking model.
  • No probability/calibration work.
  • No mass analyzer enable/disable change.
  • No automatic source-code edits in this issue.
  • No claim that every rule under MA0*, RCS*, S* or CA* has one semantic class.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions