direct: store serialized_dashboard in state as a content hash

[Sankalp-Mittal]

Summary

Store a dashboard's serialized_dashboard in direct-engine state as a sha256_hashed_in_state:<hex> content hash instead of the full multi-MB blob. The saved copy is only ever compared for equality (never read back), so a hash is sufficient and the state file stays small. The full contents are still sent to the API on every deploy.

This reimplements the approach from #5609 on current main.

What's included

• libs/hash — new OfJSON helper (deterministic sha256 of a value's JSON), with libs/cache rerouted through it (previously a private fingerprintToHash).
• hashed_in_state lifecycle rule — new []string field on ResourceLifecycleConfig, declared for dashboards.serialized_dashboard in resources.yml.
• CompactState / hashStateValue (state_compaction.go) — replace declared fields with content hashes; idempotent, never mutates the caller's value.
• Wiring — compaction applied on every state write (apply.go via compactAndSaveState, bind.go, migrate/build_state.go) and on all three diff sides in bundle_plan.go (saved, local config, remapped remote), so comparisons stay hash-vs-hash.
• Docs + unit tests — hashed_in_state section in the dresources README; unit tests for libs/hash, CompactState, and the dashboard wiring.
• Acceptance test — new bundle/resources/dashboard-state-sha case: verifies the state file holds the hash while the API receives the full content, across create / re-plan (no-op) / edit / update. Runs on the direct engine with READPLAN in ["", "1"].
• Golden outputs regenerated — existing dashboard / migrate / bind goldens updated for the new hashed state format. A [[Repls]] rule in acceptance/bundle/test.toml masks the hash as [HASH] (with Distinct, so equal hashes collapse to one token and a differing remote gets another) at Order = -1, ahead of the generic numeric/id rules that would otherwise nibble hex out of the digest.

Notes

• hashed_in_state is orthogonal to ignore_remote_changes; serialized_dashboard needs both, for independent reasons (large blob / server normalizes the content).
• Legacy full-content state is hashed on read for comparison and rewritten compactly on the next save — no state version bump.

This pull request and its description were written by Isaac.

[github-actions]

Approval status: pending

/acceptance/bundle/ - needs approval

20 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @andrewnester, @shreyas-goenka, @anton-107, @lennartkats-db

/bundle/ - needs approval

10 files changed
Suggested: @denik
Also eligible: @pietern, @janniklasrose, @andrewnester, @shreyas-goenka, @anton-107, @lennartkats-db

General files (require maintainer)

6 files changed
Based on git history:

• @denik -- recent work in bundle/direct/, bundle/direct/dresources/, bundle/migrate/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.
See OWNERS for ownership rules.}

[github-actions]

An authorized user can trigger integration tests manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

• PR number: 6103
• Commit SHA: 99f0fcff53d73121e448dea7cb2840ac1f2d82e6

Checks will be approved automatically on success.

[Sankalp-Mittal]

Superseded by #6105, which runs from a branch on databricks/cli directly (this fork-based PR couldn't get CI's OIDC/JFrog token). Closing in favor of #6105.