feat: add Docker sandbox smoke tests for Claude, Codex, and build workloads#6145
Merged
Conversation
…kloads Add three new smoke test workflows for the Docker sandbox (default AWF mode): - smoke-docker-sbx-claude.md: Docker sbx + Claude engine (haiku-4-5) - smoke-docker-sbx-codex.md: Docker sbx + Codex engine (gpt-5.4) - smoke-docker-sbx-build-test.md: Docker sbx + Node.js/Go build & test workloads These mirror the gVisor smoke tests to provide a comparison baseline and ensure Docker sandbox continues to work across engines and workloads. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds Docker sandbox smoke-test baselines across Claude, Codex, and build workloads.
Changes:
- Adds engine-specific connectivity, MCP, file I/O, and isolation tests.
- Adds Node.js and Go build/test validation.
- Adds safe-output reporting and token-usage checks.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/smoke-docker-sbx-codex.md |
Adds the Codex smoke workflow. |
.github/workflows/smoke-docker-sbx-claude.md |
Adds the Claude smoke workflow. |
.github/workflows/smoke-docker-sbx-build-test.md |
Adds multi-ecosystem build smoke tests. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 5
- Review effort level: Medium
| @@ -0,0 +1,263 @@ | |||
| --- | |||
Comment on lines
+94
to
+101
| echo "::group::npm ci" | ||
| npm ci 2>&1 | tail -5 | ||
| NPM_CI_EXIT=$? | ||
| echo "::endgroup::" | ||
|
|
||
| echo "::group::npm run build" | ||
| npm run build 2>&1 | tail -5 | ||
| NPM_BUILD_EXIT=$? |
Comment on lines
+114
to
+116
| echo "::group::npm test" | ||
| npx jest --ci --forceExit --maxWorkers=2 --testPathPattern='squid-config|docker-manager|logger' 2>&1 | tail -20 | ||
| TEST_EXIT=$? |
Comment on lines
+129
to
+131
| echo "::group::Clone Go test repo" | ||
| git clone --depth 1 https://github.com/Mossaka/gh-aw-firewall-test-go.git /tmp/test-go 2>&1 | tail -3 | ||
| CLONE_EXIT=$? |
| id: net-check | ||
| run: | | ||
| echo "::group::Blocked domain test" | ||
| BLOCKED_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 https://example.com 2>/dev/null || echo "000") |
Contributor
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
Contributor
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new smoke test workflows for the Docker sandbox (default AWF mode, no gVisor) to provide a comparison baseline and ensure Docker sandbox continues working across engines and workload types.
New Workflows
smoke-docker-sbx-claude.mdsmoke-docker-sbx-codex.mdsmoke-docker-sbx-build-test.mdDesign
These mirror the gVisor smoke tests (
smoke-gvisor-claude,smoke-gvisor-codex,smoke-gvisor-build-test) to allow direct comparison of Docker sandbox vs gVisor behavior. Each has its ownlabel_commandtrigger for targeted testing.Note
Lock files not yet compiled — will need
gh aw compilebefore these can run.