> For the complete documentation index, see [llms.txt](https://docs.digitalapi.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.digitalapi.ai/api-gateway/reference/troubleshooting.md). # Troubleshooting This chapter pairs the failure modes you are most likely to hit in the first month with their causes and one- or two-step fixes. The symptom-cause-fix table is for quick lookup; the detailed entries below it expand the same set of issues with the reproduction notes and the navigation path you need to resolve them. Coverage spans the surfaces introduced earlier in the guide: APIs (`Add API` permission, missing routes, no traffic reaching the backend), upstreams (the Add Upstream wizard, health-check failures, egress reachability), plugins (execution-order surprises), Custom Widgets (empty widgets), and Setup (invitation emails that do not arrive, role assignment caching). If a symptom is not in this chapter, jump to the matching chapter for the surface you are working on. The *Verify* sections in each task subsection are deliberately written to catch the failure modes that do not warrant their own troubleshooting entry. For anything still unresolved, the contact-support and account-manager channels listed under Additional resources are the next stops. ### Symptom-cause-fix table | Symptom | Likely cause | What to do | | ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `/welcome` renders the marketing page | Session expired or token cache cleared. | Sign in again. If a fresh sign-in still lands on the marketing page, the SPA's auth check is failing: check that the email is provisioned in the organisation. | | `Add API` button is disabled | The signed-in user lacks `api.create`. | Have an admin grant a role with `api.create`; refresh after the role is assigned. | | `Add Upstream` wizard rejects **Next** on the Overview step | A required field is empty. **Name** and **Environment** are both required on Overview before Next is enabled. | Fill **Name** and pick an environment from **Select an API environment**; **Next** becomes enabled. | | Upstream health check fails for a backend that responds to direct curl | Helix dials the host:port from its own egress; the backend may be reachable from your network but not from Helix's. | Confirm the backend is internet-reachable (or that the Helix deployment is in-network). Allow-list Helix's egress IPs at the backend's edge if needed. | | A custom widget shows ‘No data’ for an API that is taking traffic | The widget's filters scope to an API, environment, or time window that has no matching traffic. | Loosen the filters on the widget (clear environment, broaden the time window) and re-save. | | Invitation email never arrives | SMTP filtered as spam, or the Helix outbound configuration misroutes. | Check the user's spam folder. Re-invite if not delivered; if repeated re-invitation also fails, escalate to Helix support. | | New role does not appear in assignment dropdowns | Client-side caching after creation. | Hard-refresh (⌘-Shift-R or Ctrl-Shift-R) the **Users** or **Groups** page. | | Revision create returns 400 with `required property 'plugins' not found` | Route objects in `routeSpec` are missing the `plugins` field. | Add `"plugins": {}` to every route object, even routes with no route-level plugins. | | Revision create returns 400 with `$.script: must be at least 10 characters long` | A route object includes `"script": ""` (empty string). | Remove the `script` field entirely. Only include it with a real Lua script (minimum 10 characters). Same applies to `service_id`: omit it, do not set to empty string. | | Deploy returns 400 with `failed to check the configuration of plugin X` | Plugin config passes service-spec validation but fails Helix runtime validation. | Fetch the plugin's schema first: `GET /plugin-schemas/{pluginName}`. Check required fields. AI plugins especially have non-obvious requirements (`ai-prompt-template` needs `templates[]` array, `ai-proxy` needs `auth`). | | Deploy returns 400 with `Upstream not configured for environment` | Upstream binding was set via `PATCH .../upstream-bindings` (replace) instead of `.../upstream-bindings/add`. The replace endpoint silently clears bindings if the payload format is unexpected. | Always use `PATCH .../upstream-bindings/add`. Verify bindings stuck: check `upstreamBindings` array in the revision response. | | Deploy returns 409 with route conflicts | Two APIs define the same URI + method combination in the same environment. | Prefix each API's routes with a unique namespace (e.g. `/open-banking/api/...`). Use `proxy-rewrite` plugin to strip the prefix before forwarding. | | Import-openapi returns 400 with `desc: must be at most 256 characters long` | The OAS `info.description` or an operation description exceeds 256 characters. | Truncate all descriptions to under 200 characters. Single line, no markdown. | | API calls return 401 | Firebase JWT token expired (lifetime: 1 hour). | Refresh the Helix Portal page in the browser to trigger token refresh. Copy the new `Authorization: Bearer` header from DevTools → Network. | | `GET /plugin-schemas` returns only AI plugins | Built-in Helix plugins (helix-auth, cors, limit-count, etc.) are available but do not appear in the schema list. | Built-in plugins work when configured; their validation happens at the Helix runtime level during deploy, not via the schema endpoint. | | `loki-logger` plugin fails validation with `property "endpoint_addrs" is required` | Config uses `endpoint_addr` (singular) instead of `endpoint_addrs` (plural, array). | Use `"endpoint_addrs": ["http://host:3100"]` not `"endpoint_addr": "..."`. | | `proxy-cache` fails with `cache_key variable $request_method unsupported` | `$request_method` is not a valid cache key variable. | Use only `$uri`, `$host`, and supported gateway variables in `cache_key`. | ### Detailed entries #### An API is registered but no traffic reaches the backend An API alone forwards nothing. A route is what actually routes traffic to your backend. Confirm in order: 1. The API has at least one route. 2. The route targets an upstream. 3. The API is deployed to an environment. 4. Consumer traffic is hitting the environment's gateway hostname (not the portal hostname). 5. The route's match rules (path, host, method) are not stricter than expected. Any one of those five missing breaks the path. #### Cannot delete a developer because their apps are still active Developer records are referenced by apps. Either delete each app (or change its developer assignment) before deleting the developer, or use the cascade-delete flow if your role includes the relevant `force-delete` permission. The audit log captures both paths. #### API detail page shows ‘No environments’ after a deployment finished The API detail page caches its environment list briefly. Wait thirty seconds and refresh. If the deployed environment still does not appear, open `/api-gateway/environments`, find the environment, and confirm the API is listed under **Deployed APIs**. If the environment list shows the API but the API detail does not, raise it with Helix support: the two views are reconciled in the background and a persistent disagreement is a bug. #### Plugin order produces unexpected behaviour Plugins at the same scope run in a documented default order. Each plugin in the catalogue ships with a default execution order (a small integer); lower numbers run first. The default order is set by the plugin's author and matches the typical chain: authentication before rate-limiting, transformation before observability, and so on. Some plugins expose an **Order** field on the attachment form, which overrides the default for that specific attachment. Plugins without an Order field run at the default order and cannot be re-ranked without editing the plugin itself. The most common surprise is auth running after rate-limit at route scope: a misconfigured Order field rate-limits anonymous traffic before authenticating it, which fills the rate-limit bucket with junk and starves real consumers. Verify the order on the route's plugins (open the route from **Configuration → Routes**); if a plugin exposes an Order field, edit it. If not, the default order applies. #### Upstream wizard's Targets step accepts no nodes The Targets step expects at least one backend node before **Next** is enabled. If the node row is filled in but **Next** stays disabled, check the **Service discovery** toggle: when enabled, manual node entry is hidden and the upstream expects discovery to populate nodes at run time. For static-node upstreams, leave service discovery off. #### API call returns 200 on API-spec update but deploy fails This is the most common trap when managing APIs via the management API. The control plane validates structural correctness (JSON schema, field types, string lengths) when you `PUT` an API spec or route spec, but plugin-level validation only runs when the Helix runtime processes the deploy. This means: 1. API-spec update: 200 OK (even with broken plugin config) 2. Route-spec update: 200 OK 3. Deploy: 400 `"failed to check the configuration of plugin X err: property Y is required"` Before deploying, validate your plugin config against the schema endpoint: `GET /api/orgs/{orgId}/plugin-schemas/{pluginName}`. AI plugins are especially prone to this because their schemas differ from generic open-source gateway documentation. Known differences from the Helix gateway open-source documentation: * `ai-prompt-template`: requires `templates` array where each entry needs `name` (string) and `template` (object with `model` and `messages[]`). * `ai-proxy`: requires `auth` field (not optional as in OSS docs). * `loki-logger`: uses `endpoint_addrs` (plural, array) not `endpoint_addr` (singular, string). * `proxy-cache`: does not support `$request_method` in `cache_key`. {% hint style="warning" %} **Pre-deploy checklist:** Before calling the deploy endpoint, verify: (1) all routes have `plugins` (even if `{}`), (2) no route includes empty-string `service_id` or `script`, (3) plugin configs match the schema from `GET /plugin-schemas/{name}`, (4) upstream bindings are set via `/add` not the base endpoint. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.digitalapi.ai/api-gateway/reference/troubleshooting.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.