direct: store serialized_dashboard in state as a content hash - #6105
Open
Sankalp-Mittal wants to merge 23 commits into
Open
direct: store serialized_dashboard in state as a content hash#6105Sankalp-Mittal wants to merge 23 commits into
Sankalp-Mittal wants to merge 23 commits into
Conversation
…ash but full content is always sent to the API
Contributor
Approval status: pending
|
Sankalp-Mittal
marked this pull request as draft
July 30, 2026 12:41
Collaborator
Integration test reportCommit: 634ec63
9 interesting tests: 4 SKIP, 3 KNOWN, 1 RECOVERED, 1 flaky
Top 3 slowest tests (at least 2 minutes):
|
Sankalp-Mittal
marked this pull request as ready for review
July 30, 2026 15:55
Empty commit to re-run the pipeline; the previous integration run failed only on gcp-linux TestFsCp* tests, which this PR does not touch (unrelated infra flake). Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On the direct engine, store a dashboard's
serialized_dashboardin state as asha256_hashed_in_state:<hex>content hash instead of the full inlined dashboard JSON. The full contents are still sent to the API on every deploy; only the state copy is compacted.Reimplements the approach from #5609 on current
main.Why
serialized_dashboardis a large blob (often multi-MB) that the direct engine only ever compares for equality — it is never read back out of state. Persisting the whole thing bloats the state file for no benefit, so a content hash is sufficient: two hashes compare equal iff the contents do.The hash has to be applied consistently. Once the saved state holds a hash, every value entering the diff must also be hashed, or a comparison becomes hash-vs-content and reports a phantom change. So compaction runs on all three diff sides (saved, local config, remapped remote) as well as on every state write.
hashed_in_stateis orthogonal toignore_remote_changes. Because the remote side is hashed too, remote drift is still detected ashash != hash, so a field can be hashed without being ignored.serialized_dashboardhappens to need both, for the independent reason that the server normalizes the content (addspageType, reorders keys) so its remote hash never matches the config hash — drift is detected viaetaginstead.Changes
hashed_in_state []stringlifecycle rule onResourceLifecycleConfig, declared fordashboards.serialized_dashboardinresources.yml.CompactState(state_compaction.go) replaces declared fields with a content hash; it is idempotent and never mutates the caller's value (the full contents are reused for the deploy API call). It only shallow-copies the state struct, so it rejects any nested path — ahashed_in_statefield must be top-level, otherwise it would be mutated through memory shared with the deploy value and silently send the hash to the API.apply.go,bind.go,migrate/build_state.go) and into all three diff sides inbundle_plan.go.libs/hash.OfJSONhelper (deterministic sha256 of a value's JSON);libs/cacheis rerouted through it, replacing its privatefingerprintToHash.Tests
bundle/resources/dashboard-state-sha(direct engine,READPLANin["", "1"]): asserts the state file holds only the hash while the API receives the full content, across create / re-plan (no-op) / edit / update.libs/hash,CompactState(including idempotency, legacy-content migration, and nested-path rejection), a config check that every declaredhashed_in_statefield is top-level, and the dashboard wiring.[[Repls]]rule inacceptance/bundle/test.tomlmasks the hash as[HASH](withDistinct, so equal hashes collapse to one token and a differing remote gets another) ahead of the generic numeric/id rules that would otherwise nibble hex out of the digest.This pull request and its description were written by Isaac.