[2026-07-09] Incident Thread #201369
Replies: 9 comments
-
UpdateApproximately 5% of GitHub Actions runs on GitHub-hosted runners are experiencing run start delays exceeding 5 minutes. A small portion of these runs may fail after extended delays. We have identified the cause and are working on a mitigation. |
Beta Was this translation helpful? Give feedback.
-
UpdateWe are continuing to work on a mitigation. |
Beta Was this translation helpful? Give feedback.
-
Update@rentziass Let's put out an impact statement first. How about: Approximately 30% of GitHub Actions runs on GitHub-hosted runners are experiencing run start delays exceeding 5 minutes. A smaller percentage of those are exhausting retries and failing to start. |
Beta Was this translation helpful? Give feedback.
-
UpdateApproximately 30% of GitHub Actions runs on GitHub-hosted runners are experiencing run start delays exceeding 5 minutes. A smaller percentage of those are exhausting retries and failing to start. We are continuing to working on infrastructure mitigations. Next update in one hour. |
Beta Was this translation helpful? Give feedback.
-
UpdateWe're seeing Actions and Pages recovery. For a period of approximate 20 minutes ~96% of GitHub Actions runs on GitHub-hosted runners were failing to start, but has now recovered and we are seeing jobs processing. GitHub pages builds were also failing during that period, but Pages are still accessible. We are continuing to monitor for full recovery. |
Beta Was this translation helpful? Give feedback.
-
UpdateWe are continuing to monitor slow recovery in Actions and Pages builds as the system works through the high volume of backlog. Customers may see a small rate of API and job failures as the system is recovering. Copilot Cloud Agent and Copilot Code Review also failed to start for approximately 30 minutes during this incident, and we are monitoring recovery. Pages were accessible throughout the incident. |
Beta Was this translation helpful? Give feedback.
-
UpdateActions, Pages builds, Copilot Cloud Agent, and Copilot Code review have all recovered and are mitigated. We are continuing to monitor to ensure full recovery, and investigating the health of the affected infrastructure. |
Beta Was this translation helpful? Give feedback.
-
Incident ResolvedThis incident has been resolved. |
Beta Was this translation helpful? Give feedback.
-
Incident SummaryOn July 9, 2026, between 03:29 UTC and 13:39 UTC, GitHub Actions experienced delayed and failed job starts on GitHub-hosted runners. The incident was caused by an unhealthy state in a backend data service responsible for provisioning hosted runners, preventing runner acquisition for a subset of workloads. During most of the incident, approximately 8% of hosted-runner jobs experienced start delays exceeding 5 minutes, while roughly 2% failed to start. At 13:39 UTC, we restored the health of the backend data replication system, allowing provisioning to recover and the accumulated workflow backlog to drain. Service performance then returned to expected levels. We are improving provisioning-service resiliency, workload distribution, and capacity balancing to reduce the likelihood and impact of similar incidents. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
❗ An incident has been declared:
Delays starting Actions runs
Subscribe to this Discussion for updates on this incident. Please upvote or emoji react instead of commenting +1 on the Discussion to avoid overwhelming the thread. Any account guidance specific to this incident will be shared in thread and on the Incident Status Page.
Beta Was this translation helpful? Give feedback.
All reactions