Runtime witness generation: turn supported OWN001/OWN014 evidence into red→green retention tests

[PhysShell]

Classification

• Change type: enhancement / static-to-runtime verification capability
• Priority: P2
• Expected value: high for flagship WPF lifetime findings and remediation proof
• Estimated effort: high
• Implementation risk: high — weak/generated tests can create false confidence
• Recommendation: YES, valuable product differentiator; schedule after alpha/release and identity foundations, not before
• Related: P-034 follow-up: verify opt-in runtime disposal-guard coverage and offer source-visible fixes #241 and docs/proposals/P-034-runtime-lifetime-guard.md
• Downstream consumer: PhysShell/OwnAudit#43

Problem

OWN001/OWN014 provide structured static evidence for resource acquisition without release and lifetime promotion/retention. OwnAudit has a runtime arm and a Fix Arm, but there is no narrow bridge that turns supported static evidence into an executable retention witness.

Today the strongest remediation statement is generally:

the static finding disappeared and no new static findings were introduced

For selected WPF/event shapes, the system can do better:

static candidate exists
runtime witness reproduces retention
patch is applied
static candidate disappears
runtime witness becomes green

That is a materially stronger proof than another generated test which merely constructs an object and congratulates itself for not throwing.

Goal

Generate conservative, reviewable runtime witness tests from supported OWN001/OWN014 evidence shapes.

A witness should attempt to reproduce the relevant lifecycle, release strong references, force bounded collection attempts, and assert whether the subscriber/owner remains alive.

Unsupported cases must produce typed refusal reasons. The generator must prefer refusal over a tautological or flaky test.

Initial supported shapes

Start narrowly with fixtures, then validate against real cases:

1. WPF Window or FrameworkElement subscribes a named instance handler to a longer-lived/static event and lacks a matching teardown release.
2. ViewModel/owner subscribes to an injected singleton/app-lifetime event source, with an explicit lifecycle method available (Dispose, Closed, Unloaded, or fixture-supplied teardown hook).
3. A matching fix adds the release at a supported teardown boundary.

Do not begin with arbitrary lambdas, unknown delegate storage, cross-thread dispatchers, unmanaged callbacks, or sources whose lifetime cannot be isolated.

Witness shape

Generated code may be equivalent to:

[TestMethod]
public void Closing_owner_releases_subscription()
{
    WeakReference weak = CreateExerciseAndReleaseOwner();

    ForceCollectionWithinBound();

    Assert.IsFalse(
        weak.IsAlive,
        "Owner remains retained after its declared lifecycle ended.");
}

CreateExerciseAndReleaseOwner() must be generated from evidence + fixture configuration, not guessed from arbitrary application startup code.

The witness metadata must record:

finding pattern/occurrence identity
evidence digest
source/member symbols
lifecycle action
long-lived source identity
expected before verdict
collection bounds/runtime/environment
limitations/refusal reason

Flakiness controls

GC tests are notorious for turning certainty into interpretive dance. Required controls:

1. Isolate the retained object creation in a non-inlined helper where necessary.
2. Remove all test-owned strong references before collection.
3. Use bounded repeated collection/finalizer cycles rather than an unbounded wait.
4. Distinguish retained, collected, inconclusive, and environment-error.
5. Pin runtime/architecture/environment metadata.
6. Run witness controls proving the harness can collect an equivalent unsubscribed object.
7. Treat dispatcher/thread/apartment requirements explicitly.
8. Never convert an inconclusive run into pass.

Generator output

Suggested output:

witness/
  <occurrence-id>/
    WitnessTests.cs
    witness.json
    README.md

witness.json should be suitable for OwnAudit remediation proof ingestion and include test/build result references and digests.

Required controls

1. Known static-event leak fixture → witness fails before fix.
2. Same fixture with matching unsubscribe → witness passes.
3. Equivalent clean control object is collectable under the same harness.
4. A remaining unrelated strong reference → witness remains failed/inconclusive, never falsely passes.
5. Unsupported inline/block lambda → typed refusal.
6. No safe lifecycle boundary → typed refusal.
7. Required STA/dispatcher environment absent → environment error/inconclusive.
8. Finding identity/evidence changes → stale witness is rejected or regenerated.
9. Generated witness compiles against the pinned fixture/project surface.
10. Re-running under the same supported environment is stable within the documented bound.

Acceptance

• Versioned witness specification and refusal vocabulary are documented.
• At least two supported WPF/event shapes generate compiling tests.
• Each supported fixture demonstrates red-before/green-after behavior.
• Harness includes a collectable negative control.
• Inconclusive/environment failures remain distinct from pass/fail.
• Witness artifacts carry finding identity, evidence and toolchain provenance.
• OwnAudit can consume witness verdicts without treating absence as pass.
• Unsupported shapes fail honestly and are measured in a coverage ledger.

Non-goals

• No claim that GC/WeakReference tests prove absence of every leak.
• No arbitrary application UI automation in the first slice.
• No automatic merge approval from a generated witness.
• No replacement of static analysis or OwnAudit runtime correlation.
• No support for every OWN001/OWN014 shape.
• No weakening of P-034 runtime guard semantics.