syntworldLet’s talk
Language: EN
All insights

Incident analysis

Railway says an intruder may have decrypted some customer environment variables

Railway says an unauthorized party may have decrypted some customer environment variables. What is confirmed, what to rotate, and where to look for misuse.

By Synt WorldPublished 7 min read

At a glance

Bottom line
If you deploy on Railway, treat every environment variable, registry credential and GitHub token you stored there as exposed. Revoke each at its issuing provider, redeploy, then check that provider's logs. Railway asks all users to do this.
Affected
A subset of users, according to Railway's initial analysis. No count has been published. Railway reports more than 2 million users overall; nothing it has published says millions had secrets taken.
Status
As of 08:01 UTC on October 22, 2026: no published root cause, exposure window or evidence of misuse. Builds were back for Enterprise and Pro; Hobby, Free and Trial were still waiting.

What Railway has said

On October 21, 2026, Railway said it had detected anomalous activity in the systems that store customer environment variables. Its investigation so far indicates that an unauthorized third party may have accessed the encrypted variables and, for some of them, obtained the decrypted contents. Railway notes those variables may hold API keys and database and registry credentials.

Railway says it believes this is part of a series of attacks on platform companies aimed at compromising AI inference providers, and that it asked those providers to revoke all keys proactively. It has not named the attacker, the other platforms or the providers. It says the impact may be limited to a subset of users, still asks everyone to roll all variables and registry credentials, and will email the customers it confirms were affected.

How the response unfolded

  1. 07:24 UTC: Railway opens a status incident, Railway Platform Outage. Builds are soon paused in all regions.
  2. 20:44 UTC: Security disclosure published on Railway's blog.
  3. 22:10 UTC: Railway starts rotating every storage bucket credential.
  4. 01:22 UTC: Railway lists 13 recovery guides.
  5. 03:32 UTC: Builds return for Enterprise workspaces.
  6. 04:17 UTC: All Railway API tokens invalidated.
  7. 06:29 UTC: Builds return for Pro workspaces.
  8. 08:01 UTC: Latest update when we checked. Hobby, Free and Trial builds still pending.

Railway has not said when the intruder got in, whether the access has ended, or how it was obtained. Treat the exposure window as unknown.

Bar chart: hours after Railway's security notice at 20:44 UTC on October 21, 2026. Bucket credential rotation started: 1.4. Enterprise builds back: 6.8. All Railway API tokens invalidated: 7.6. Pro builds back: 9.8.
Hour 0 is the disclosure, 20:44 UTC on October 21. Until builds came back for your plan, you could revoke a key at its provider but not ship the replacement. Hobby, Free and Trial were still waiting 11.3 hours in.

What Railway reset for you, and what it did not

  • Storage buckets: credentials for every bucket in every environment were regenerated. Services pick them up only after a redeploy. The resets show in your audit log under the actor type Railway Staff.
  • GitHub: Railway invalidated the account-connection tokens it held. Reconnect GitHub and check that autodeploy is still on.
  • GitHub registry tokens: personal access tokens stored for ghcr.io were deleted from Railway, but Railway says they still work on GitHub until you revoke them there. Do that first.
  • Railway API tokens: all invalidated, so integrations need new ones. Railway also invalidated OAuth sessions, signing some users out.

Everything else is on you: every value you put in a variable and every credential for an outside service. Railway asked AI providers to revoke keys. Don't count on it.

What the record does not support yet

No count of affected customers. No root cause. No start or end time for the access. And we found no report, from Railway or any provider, of stolen keys being used. That only means nothing has been reported. Your provider logs are where you find out.

About "millions of developers": Railway's homepage says 2M+ developers, and its January 2026 funding announcement said more than 2 million users. That is the size of the platform by Railway's own count, not a victim count. Press coverage so far, such as Cyber Security News, repeats Railway's statement and adds nothing independent on scope or entry point.

Find every secret that lived in Railway

You can't rotate what you don't know you stored. Railway's CLI lists every variable a service sees; in the current CLI, --json shows sealed variables by name with a null value. The loop below keeps names and drops values, so the file is safe to share with your team.

# Sign in again first: Railway invalidated sessions and API tokens.
# Run in a directory linked to the project (railway link). Writes variable
# NAMES only, never values. Repeat per project, and list every environment
# (including PR environments) and every service.
for env in production staging; do
  for svc in api worker; do
    railway variable list --service "$svc" --environment "$env" --json |
      jq -r --arg e "$env" --arg s "$svc" '
        to_entries[]
        | select(.key | startswith("RAILWAY_") | not)
        | [$e, $s, .key, (if .value == null then "sealed" else "readable" end)]
        | @tsv'
  done
done > railway-variable-inventory.tsv

The CLI misses some places. Check registry credentials in each service's Source settings, PR environments, which inherit copies of variables, and every copy outside Railway: CI, laptops, BI tools, a second host. Rotating a key means updating all of them.

Rotate in this order

Revoke at the issuing provider. Editing a Railway variable without killing the old value at its source closes nothing, and containers read variables only at startup, so each change needs a redeploy.

  • AI inference keys first, since Railway says the attackers were after AI inference providers: OpenAI, Anthropic, xAI, OpenRouter, and any gateway you run. Give replacements a spend limit and an expiry where the provider allows.
  • Cloud keys. Deactivate AWS access keys rather than delete them. For Google service account keys, tokens already minted stay valid for up to an hour after you disable the key; disable the service account for an immediate cut.
  • Stripe secret and restricted keys and webhook signing secrets. For a confirmed leak, Railway's guide says expire the old key now, with no overlap window.
  • Databases. On Railway templates, use Regenerate; Railway warns that editing the password variable or running ALTER USER by hand leaves the two out of sync. Turn off Public Access unless you need it: with it on, a leaked password works from anywhere.
  • GitHub personal access tokens and deploy keys.
  • Application secrets: session and JWT signing keys, OAuth client secrets, encryption keys. A leaked signing key lets someone mint sessions as your users. If an encryption key sat next to the database URL for the data it protects, assume that data is readable.

Revoking before you can redeploy breaks your service until the new value ships. For keys that spend money or reach customer data, take the outage. Railway's PostgreSQL guide makes the same call.

Look for misuse where the keys work

Railway's audit log records actions by users, Railway staff and Railway's system. Railway hasn't said whether the intruder's access appears there, so a clean audit log proves little. With no published window, search each provider's logs as far back as they go, up to the moment you rotated.

  • AWS: look up each exposed access key ID in CloudTrail event history in every region, since it is per region. Look for new users, keys, roles and policies, instance launches and unused regions, then compare Cost Explorer with last month.
  • Google Cloud: query Cloud Audit Logs by the service account's email for new keys, IAM bindings and impersonation. Data Access logs are off by default for most services, so reads of your data may be unrecorded. Railway says compute, GPU and Vertex AI spend are the common targets.
  • AI providers: group usage by key. Railway notes that a key suddenly calling a model you never use is a strong signal. Export usage before deleting a key.
  • Stripe keeps request logs for rotated and deleted keys. Check for unfamiliar IPs, then for webhook endpoints, bank accounts and team members you didn't add.
  • Databases: look for roles, extensions and scheduled jobs you didn't create. PostgreSQL doesn't log successful connections by default, so turn that on now.
  • GitHub: export the security log and organization audit log before revoking, then review tokens, SSH and deploy keys, and webhooks.

Without static outbound IPs, Railway services leave from shared addresses, so an unfamiliar IP is a lead, not proof, as Railway's own Google Cloud guide says.

Make the next PaaS breach smaller

The uncomfortable part: encryption at rest only goes so far. A platform that injects secrets into your containers has to be able to decrypt them, so anyone who controls that path can too. Sealed variables don't change this. Railway describes them as write-only in the UI and API, which stops a leaked dashboard session, but Railway still injects them, and its notice says to roll all variables.

Short-lived credentials beat stored ones. Where a platform gives each workload a signed identity token, AWS and Google Cloud can exchange it for credentials that expire on their own, typically within an hour. Railway's own guide says its built-in secrets suit teams that don't need automatic rotation or short-lived dynamic secrets, and we found no workload identity feature in its documentation index. There you can't remove the last secret, but you can make it the only one.

  • One bootstrap secret, not forty. Keep real secrets in a manager you control and give each service one credential that reads only its own path, pinned to your static outbound IPs (a Railway Pro feature). If it leaks, you rotate one value, and the manager's access log shows exactly what was read.
  • Fetch at runtime. A service that refreshes secrets from a manager can take a rotated value without a build. This week, rotating variables meant waiting on a build pipeline that was paused for most of a day.
  • One key per service and environment, so a leak stays in one place and provider logs tell you which service made a request. Give PR environments test credentials.
  • Cap damage at the provider: spend limits on AI keys, Stripe restricted keys, least-privilege IAM, IP allowlists where offered, no public database access.

Railway's response has been concrete: 13 guides within about four hours and forced resets where it could do them. It still can't rotate your Stripe key or tell you whether someone spent your OpenAI credits overnight. That part is yours. Do it today, not when the email arrives.

Sources

  1. Railway: Security Disclosure: Wednesday October 21st
  2. Railway Status: Railway Platform Outage (incident QH8L8B3E)
  3. Railway: Security Disclosure Recommendation: Redeploy Buckets and Reconnect GitHub After the Credential Reset
  4. Railway: Security Disclosure Recommendation: Rotate PostgreSQL Database Credentials
  5. Railway: Security Disclosure Recommendation: Check for Anomalous AWS Access
  6. Railway: Security Disclosure Recommendation: Check for Anomalous GCP Access
  7. Railway: Security Disclosure Recommendation: Check for Anomalous GitHub Access
  8. Railway: Security Disclosure Recommendation: Check for Unexpected Anthropic Usage
  9. Railway: Security Disclosure Recommendation: Check for Unexpected OpenAI Usage
  10. Railway: Security Disclosure Recommendation: Check for Unexpected OpenRouter Usage
  11. Railway: Security Disclosure Recommendation: Check for Unexpected Usage in Stripe
  12. Railway Docs: Managing Secrets on Railway
  13. Railway Docs: railway variable
  14. GitHub, railwayapp/cli: src/commands/variable.rs
  15. Railway Docs: Audit Logs
  16. Railway Docs: Static Outbound IPs
  17. Railway Docs: documentation index (llms.txt)
  18. Railway: homepage ("Join 2M+ developers")
  19. PR Newswire: Railway Raises $100 Million Series B As AI Pushes Today's Cloud Infrastructure Past Its Limits
  20. Cyber Security News: Railway Security Incident: Hackers May Have Accessed Customer Environment Variables

Concerned about your own exposure?

Discuss your audit

Privacy settings

The site works without optional analytics. We store your choice for 180 days.

In supported regions, we count page deliveries without visitor identifiers. Reject all also stops future optional counts.

Privacy notice