refactor(server): decouple K8s ServiceAccount bootstrap from selected compute driver config

[elezar]

Description

K8s ServiceAccount bootstrap currently derives its namespace and service-account settings from [openshell.drivers.kubernetes] whenever the gateway has gateway_jwt configured and detects that it is running in-cluster.

That assumes the Kubernetes compute driver is the source of sandbox bootstrap identity configuration. With named remote compute drivers, an in-cluster gateway may select a non-Kubernetes driver but still hit the bootstrap path and require [openshell.drivers.kubernetes].

Context

PR #1974 normalizes selected compute-driver config acquisition so unselected driver tables are no longer validated during startup. That makes the ServiceAccount bootstrap path stand out as a separate consumer of Kubernetes driver config, outside the selected driver acquisition flow.

Related issues:

• refactor(server): normalize compute driver acquisition path #1949 normalized compute driver acquisition.
• refactor: make sandbox readiness gateway-owned across compute drivers #1951 tracks gateway-owned readiness across drivers.
• refactor: align driver ownership of gateway callback listeners #1952 tracks gateway-owned callback listeners.

Proposed Direction

Make K8s ServiceAccount bootstrap configuration explicit and independent from the selected compute driver, or only enable the current Kubernetes-driver-derived bootstrap behavior when Kubernetes is the selected driver.

The implementation should decide how an in-cluster gateway using a named remote driver configures or disables ServiceAccount bootstrap without requiring an unrelated [openshell.drivers.kubernetes] table.

Definition of Done

• Define the ownership model for ServiceAccount bootstrap namespace and service-account settings.
• Avoid requiring [openshell.drivers.kubernetes] when the selected compute driver is a named remote driver.
• Preserve current Kubernetes-driver behavior for in-cluster gateways that rely on ServiceAccount bootstrap.
• Add tests for Kubernetes selected, remote driver selected in-cluster with bootstrap enabled, and remote driver selected with bootstrap disabled or explicitly configured.
• Update gateway configuration documentation if new config fields are introduced.

[TaylorMutch]

🏗️ build-plan

Implementation Plan

Issue type: refactor
Complexity: Medium
Confidence: High — clear path; the main design choice is to reuse existing gateway-level fields instead of adding a new TOML table.

Summary

Decouple K8s ServiceAccount bootstrap from [openshell.drivers.kubernetes] by resolving the bootstrap namespace and service account from gateway-owned configuration. Preserve current Kubernetes-driver behavior by falling back to the selected Kubernetes driver config when the Kubernetes driver is selected. For named remote drivers, enable ServiceAccount bootstrap only when [openshell.gateway] explicitly supplies both sandbox_namespace and service_account_name; otherwise leave the bootstrap path disabled without requiring or validating an unrelated Kubernetes driver table.

Scope

• crates/openshell-server/src/lib.rs: replace kubernetes_config_for_k8s_sa_bootstrap with a small bootstrap-config resolver that takes the selected compute driver and returns either explicit bootstrap config or disabled state.
• crates/openshell-server/src/config_file.rs: no schema changes expected; add or adjust tests only if the resolver needs additional helper access to existing gateway fields.
• deploy/helm/openshell/templates/gateway-config.yaml: render service_account_name at gateway scope so Helm output reflects gateway ownership while preserving the existing Kubernetes driver value for compatibility.
• docs/reference/gateway-config.mdx: document that sandbox_namespace and service_account_name also control K8s ServiceAccount bootstrap when the gateway is in-cluster with sandbox JWT issuing enabled.
• docs/reference/sandbox-compute-drivers.mdx: clarify Kubernetes fallback behavior and named remote driver bootstrap requirements.
• architecture/gateway.md: update the authentication overview to describe the gateway-owned bootstrap identity check.

Implementation Steps

1. Introduce an internal K8sSaBootstrapConfig with namespace and service_account_name, plus a resolver that accepts the selected driver kind and the loaded ConfigFile.
2. For selected kubernetes, resolve from the merged Kubernetes driver config to preserve existing deployments and inherited gateway defaults.
3. For named remote drivers, resolve only from explicit [openshell.gateway].sandbox_namespace and [openshell.gateway].service_account_name; if neither is set, return disabled; if only one is set, return a config error.
4. Keep Docker, Podman, and VM gateways from implicitly enabling this bootstrap path unless a future issue explicitly defines that behavior.
5. Update gateway startup to call the resolver after compute-driver selection and only construct K8sServiceAccountAuthenticator when it returns a config.
6. Update Helm rendering and docs to make the ownership model visible.
7. Run focused unit tests, then the repository pre-commit lane before committing.

Test Plan

• Unit tests: Add resolver tests in crates/openshell-server/src/lib.rs for Kubernetes selected, named remote selected with explicit bootstrap config, named remote selected with bootstrap disabled, and named remote selected with partial bootstrap config rejected. Include a regression case showing a named remote selection does not require [openshell.drivers.kubernetes].
• Integration tests: N/A — the change is startup config resolution and authenticator wiring; no live Kubernetes API is needed for the resolver behavior.
• E2E tests: N/A unless implementation touches Helm/E2E fixtures. This is not expected to require a sandbox e2e lane.

Risks & Open Questions

• Reusing the existing gateway-level service_account_name avoids schema churn, but human review should confirm that a dedicated [openshell.gateway.k8s_sa_bootstrap] table is not preferred for long-term clarity.
• Named remote drivers that rely on this bootstrap path must create sandbox pods using the same namespace and service account configured on the gateway.
• LSM compatibility impact is not expected; the change does not touch process identity, /proc, binary execution, or inter-process visibility.

Documentation Impact

• Update docs/reference/gateway-config.mdx, docs/reference/sandbox-compute-drivers.mdx, and architecture/gateway.md.
• No docs/index.yml navigation change expected.
• No gateway TOML schema field is added, but existing fields gain clarified behavior.

---

Revision 1 — initial plan

[elezar]

The build plan's activation model needs adjustment before implementation proceeds.

Problem with the current plan

The proposed activation condition — enable bootstrap when both sandbox_namespace and service_account_name are explicitly set — infers intent from config presence rather than expressing it directly. These fields already exist as gateway-level inheritance defaults for the K8s driver; repurposing them as bootstrap activation signals would be subtle and surprising.

More fundamentally: bootstrap is a K8s API operation (TokenReview, Pod lookup, Sandbox CR resolution). It only makes sense when sandboxes are K8s pods. The natural gate is driver selection, not config presence.

Revised direction

Bootstrap activation should be owned by driver selection:

• K8s driver selected + in-cluster (KUBERNETES_SERVICE_HOST set) + gateway_jwt configured → bootstrap enabled, namespace and service account sourced from the already-resolved K8s driver config
• Named remote, Docker, Podman, VM → bootstrap never enabled, no error

This also eliminates kubernetes_config_for_k8s_sa_bootstrap — it exists solely because the bootstrap block currently does a second independent parse of [openshell.drivers.kubernetes]. Once the bootstrap block is gated on K8s driver selection, it can read from the driver config that was already resolved during startup.

Revised Definition of Done

• Bootstrap is enabled if and only if the Kubernetes driver is selected, the gateway is running in-cluster, and gateway_jwt is configured
• Named remote, Docker, Podman, and VM drivers never enable bootstrap — defer named remote opt-in to a follow-on issue if ever needed
• Delete kubernetes_config_for_k8s_sa_bootstrap; source namespace and service account from the already-resolved K8s driver config
• No new TOML fields or schema changes
• Tests: K8s selected + in-cluster + gateway_jwt → bootstrap enabled; K8s selected + not in-cluster → skipped; named remote + in-cluster + gateway_jwt → skipped without error
• Update docs/reference/gateway-config.mdx and docs/reference/sandbox-compute-drivers.mdx to document that bootstrap is gated on Kubernetes driver selection
• Update architecture/gateway.md to reflect the corrected activation model

The open question about a [openshell.gateway.k8s_sa_bootstrap] sub-table is explicitly out of scope — close it as deferred.

[github-actions]

This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.

[grs]

I believe #2460 (still in draft) would address the issue by only enabling the kubernetes bootstrap when that is used as the compute driver:

• Kubernetes ServiceAccount bootstrap identity resolution is moved into openshell-driver-kubernetes.
• ComputeRuntime::new_kubernetes(...) wires KubernetesSupervisorBootstrapIdentityProvider.
• Remote/out-of-tree driver construction does not get that provider, so the gateway authenticator chain has no Kubernetes SA bootstrap path for non-Kubernetes selected drivers.
• The gateway still owns the public RPCs and stream mechanics, but delegates Kubernetes token/pod/Sandbox validation through the selected runtime’s bootstrap identity provider.