fix(api-proxy): fall back to next alias candidate when Copilot rejects model for endpoint#6134
Conversation
When the Copilot API rejects a resolved model with "not accessible via the /chat/completions endpoint" (e.g. gpt-5.4-mini), the api-proxy now falls back to the next ranked candidate from the alias resolution rather than returning the 400 error to the client. Root cause: The `summarization` alias resolves via the `gpt-5-mini` sub-alias which uses pattern `copilot/gpt-5*mini*`. This matches both `gpt-5-mini` and `gpt-5.4-mini` from the Copilot model list. The version sorter picks `gpt-5.4-mini` (v5.4 > v5.0) as the top candidate. But the Copilot API only supports `gpt-5.4-mini` via /responses, not via /chat/completions. The result is a 400 that previously was not recognized and was forwarded directly to the client (Copilot CLI), causing all three Test Coverage Reporter runs to fail with 0 AIC. Changes: - model-resolver.js: `_resolveAliasPatterns` now returns the full ranked `candidates` array alongside `resolvedModel` so callers have access to fallback options. - model-body-rewriter.js: propagates `candidates` from resolution result. - model-config.js: body transform now accepts (body, req) and stores `req.awfModelCandidates` for use during retry. - body-handler.js: passes `req` to `bodyTransform(body, req)`. - upstream-response.js: adds `MODEL_ENDPOINT_BLOCKED_PATTERN` matching "not accessible via the ... endpoint" errors; adds `parseModelEndpointBlockedFromBody`; threads `onModelEndpointBlockedRetry` callback through `handleUpstreamResponse` and `handle400WithRetry`. - upstream-retry.js: new branch (b) handles endpoint-blocked 400s by calling `onModelEndpointBlockedRetry()` before the transient retry. - upstream-http.js: implements `onModelEndpointBlockedRetry` — reads `req.awfModelCandidates`, finds current model in the body, picks the next ranked candidate, rewrites the body, and retries upstream. - model-api-mapping.json + docs/model-api-mapping.json: updated notes for gpt-5.4 family to document the Copilot /chat/completions restriction. - upstream-retry.test.js: adds 4 tests for endpoint-blocked fallback. - model-resolver.test.js: adds test verifying `candidates` is returned.
There was a problem hiding this comment.
Pull request overview
This PR improves the api-proxy’s Copilot model-alias handling by detecting “model not accessible via endpoint” 400s and retrying the same request with the next ranked alias candidate, preventing hard failures when the highest-semver candidate is responses-only on Copilot.
Changes:
- Add endpoint-blocked (“not accessible via the … endpoint”) detection and a no-delay retry path that falls back to the next alias candidate for Copilot.
- Thread ranked alias candidates through model resolution/body-rewrite so retry logic can reuse them without re-resolving.
- Document the Copilot
/chat/completionsrestriction forgpt-5.4-miniin model mapping notes.
Show a summary per file
| File | Description |
|---|---|
| docs/model-api-mapping.json | Documents Copilot endpoint restriction + fallback behavior for gpt-5.4-mini. |
| containers/api-proxy/model-api-mapping.json | Mirrors the same restriction + fallback note for the runtime mapping file. |
| containers/api-proxy/upstream-retry.test.js | Adds unit coverage for the endpoint-blocked retry branch. |
| containers/api-proxy/upstream-retry.js | Adds endpoint-blocked retry branch ahead of transient model-not-supported retry. |
| containers/api-proxy/upstream-response.js | Buffers relevant 400 bodies and wires endpoint-blocked parsing + callback into retry handling. |
| containers/api-proxy/upstream-http.js | Implements request-body rewrite + resend using next ranked candidate. |
| containers/api-proxy/model-resolver.test.js | Verifies ranked candidates are returned to support fallback. |
| containers/api-proxy/model-resolver.js | Returns full ranked candidate list alongside resolvedModel. |
| containers/api-proxy/model-body-rewriter.js | Propagates candidates from resolver into rewrite result. |
| containers/api-proxy/model-config.js | Stores ranked candidates on req during body transform for later retry use. |
| containers/api-proxy/body-handler.js | Passes req into bodyTransform to allow storing per-request state. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 11/11 changed files
- Comments generated: 2
- Review effort level: Low
| // Determine which model was sent in the current body. | ||
| const parsed = parseBodyAsObject(body); | ||
| const currentModel = parsed && parsed.model; | ||
| if (!currentModel) return false; | ||
|
|
||
| const currentIdx = candidates.indexOf(currentModel); | ||
| if (currentIdx < 0 || currentIdx >= candidates.length - 1) return false; | ||
|
|
||
| const nextModel = candidates[currentIdx + 1]; | ||
|
|
||
| // Rewrite the body with the next candidate. | ||
| const newParsed = parseBodyAsObject(body); | ||
| if (!newParsed) return false; | ||
| newParsed.model = nextModel; | ||
| const newBody = Buffer.from(JSON.stringify(newParsed), 'utf8'); | ||
|
|
||
| // Update the candidates list so if the next model also fails we can | ||
| // continue falling back (by shifting the current index forward). | ||
| sendUpstreamRequest(requestHeaders, { | ||
| body: newBody, targetHost, upstreamPath, req, res, provider, requestId, startTime, span, requestBytes, | ||
| hasRetried, | ||
| modelNotSupportedRetryCount, | ||
| }); | ||
| return true; | ||
| }, |
| async function transformRequestBody(body, provider, req, requestId, bodyTransform) { | ||
| if (bodyTransform && (req.method === 'POST' || req.method === 'PUT' || req.method === 'PATCH')) { | ||
| const transformed = await bodyTransform(body); | ||
| const transformed = await bodyTransform(body, req); |
|
@pelikhan can you take a look at this? |
|
⏳ Copilot review left inline comments. @copilot To proceed:
|
|
✅ Copilot review passed with no inline comments. @copilot Add the |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
|
✅ Smoke Claude passed |
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
✅ Build Test Suite completed successfully! |
|
✅ Contribution Check completed successfully! |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
📰 DEVELOPING STORY: Smoke Docker Sbx reports failed. Our correspondents are investigating the incident... |
|
✅ Smoke Gemini completed. All facets verified. 💎 Connectivity check failed: 000 (Exit 7). File writing passed. Searching for PRs in git log. |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
🔌 Smoke Services — All services reachable! ✅ |
|
🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅ |
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓 |
|
❌ Security Guard failed. Please review the logs for details. |
|
✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓 |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
Thanks for the clear explanation of the endpoint-blocked fallback and for including focused tests and documentation updates. One contribution-guideline item remains: please reference any related issue in the PR description, as required under Pull Request Process → Pull request requirements in CONTRIBUTING.md. If no related issue exists, a brief note stating that would make the linkage explicit. Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Claude Engine Validation
Overall Result: PASS ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test Results
Overall: PASS PR author: Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Services Connectivity
Overall: FAIL — Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Copilot BYOK (Direct) ModeResults
Status: PASS (core path working) Running in direct BYOK mode via Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
|
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test: Copilot PAT Auth
Overall: PASS 🎉 Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Smoke Test Results: Gemini Engine
Overall Status: FAIL Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
|
Test comment Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
🔍 Smoke Test: API Proxy OpenTelemetry Tracing
Overall: All 5 scenarios pass. 🎉 Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
Chroot Version Comparison Results
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
|
Merged PRs:\n- compiled workflows\n- compiled workflows\nChecks:\n- GitHub query: ✅\n- Browser: ✅\n- Smoke file: ✅\n- Build: ✅\nOverall: PASS Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"
- "registry.npmjs.org"See Network Configuration for more information.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "awmgmcpg"See Network Configuration for more information.
|
The Test Coverage Reporter workflow was failing all runs with 0 AIC because the
summarizationalias resolved togpt-5.4-mini(highest semver fromcopilot/gpt-5*mini*), which the Copilot API permanently rejects on/chat/completionswith400 model "gpt-5.4-mini" is not accessible via the /chat/completions endpoint. The error was unrecognized and forwarded directly to the Copilot CLI.Root cause
The
gpt-5-minialias patterncopilot/gpt-5*mini*matches bothgpt-5-miniandgpt-5.4-mini. Version sorting picksgpt-5.4-mini(5.4 > 5.0) as the winner. The Copilot API lists it in the models catalogue but restricts it to/responsesonly.Changes
New: endpoint-blocked detection + per-request candidate fallback
upstream-response.js: addsMODEL_ENDPOINT_BLOCKED_PATTERN(/not accessible via the .+? endpoint/i) andparseModelEndpointBlockedFromBody; buffers 400s for this case; threadsonModelEndpointBlockedRetrycallback through tohandle400WithRetryupstream-retry.js: new branch (b) firesonModelEndpointBlockedRetry()before the existing transient-model-not-supported branch — permanent endpoint restriction, no delayupstream-http.js: implementsonModelEndpointBlockedRetry— readsreq.awfModelCandidates, finds current model in body, picks next ranked candidate, rewrites body, retriesRanked candidates wired through resolution pipeline
model-resolver.js(_resolveAliasPatterns): returnscandidates: unique(full sorted list, e.g.[gpt-5.4-mini, claude-haiku-4.5]) alongsideresolvedModelmodel-body-rewriter.js: propagatescandidatesfrom resolution resultmodel-config.js: body transform signature becomesasync (body, req)— storesreq.awfModelCandidateson the request for later use by retry logicbody-handler.js: passesreqtobodyTransform(body, req)Documentation
model-api-mapping.json(bothcontainers/api-proxy/anddocs/): notes added togpt-5.4*family documenting the Copilot/chat/completionsrestriction and the proxy's fallback behaviourFallback flow (after fix)