Error 503 and timeouts

Most calls are answered from our own database, even when the public registers are down. Some enrichments fetch data live from Datafordeleren and other public sources, and they can answer 503 when the source does not respond. This guide shows which ones, what the response contains, and how to set timeouts and retries.

What depends on what#

Address data is updated from Datafordeleren in the background. If Datafordeleren is down you still get answers, but changes from the last few hours may be missing until the sync is back.

upstream_unavailable#

Datafordeleren has planned service windows and occasional outages. During a service window it rejects every key, and then we cannot fetch data that is not already in our cache. The response is:

HTTP/2 503
retry-after: 60
cache-control: no-store
content-type: application/json

{
  "error": {
    "code": "upstream_unavailable",
    "message": "An upstream data register is temporarily unavailable. Retry in a minute.",
    "details": null
  }
}

The same response comes back if Datafordeleren does not answer within our own timeout.

What to do:

• Retry after the number of seconds in Retry-After. Browser code can read the header too, because the API exposes it through CORS. A service window can last from minutes to hours, so let the wait grow after a few attempts.
• Show the user the address without the enrichment, and fetch it later. Address data is still available.
• If you fetched the enrichment for the same address within the last 24 hours, it comes from the cache and works during the outage too.
• See the service status at /status.

upstream_busy#

{
  "error": {
    "code": "upstream_busy",
    "message": "Enrichment is briefly rate-limited to protect the upstream register. Retry in a moment.",
    "details": null
  }
}

All customers share one key to the public registers. When many enrichment calls for addresses that are not cached arrive at the same time, we hold back briefly. The response is 503 with Retry-After: 2. Wait two seconds and retry. If you need to enrich many addresses, do it at an even pace rather than in large parallel waves.

502 and 504 from the proxy#

A 502 or 504 without a JSON body comes from the proxy in front of the API: a timeout on a slow request or a brief moment during a deployment. The body is an HTML page, so your error handling must not assume JSON. Treat them as a 503 without Retry-After: wait a few seconds and retry. An error from the API itself, a 500 included, always has a JSON body with a code.

Timeouts in your client#

Without a timeout a hanging connection can block a thread or a worker for a long time. Always set both a connect timeout and a read timeout:

An example with httpx, which has separate timeouts for connecting and reading:

import os

import httpx

timeout = httpx.Timeout(connect=5.0, read=15.0, write=10.0, pool=5.0)
with httpx.Client(
    base_url="https://api.danadresse.dk",
    headers={"X-Api-Key": os.environ["DANADRESSE_API_KEY"]},
    timeout=timeout,
) as client:
    r = client.get("/postnumre/8000")
    r.raise_for_status()
    print(r.json()["navn"])

Retries for enrichments#

This function fetches an enrichment and retries on 502, 503 and 504 and on timeouts. It uses Retry-After when the header is present, and a growing wait otherwise. 4xx errors are not retried, because they do not go away by themselves. Enrichments need a live key:

import os
import random
import time

import httpx

API = "https://api.danadresse.dk"
RETRY_STATUS = {502, 503, 504}

client = httpx.Client(
    base_url=API,
    headers={"X-Api-Key": os.environ["DANADRESSE_API_KEY"]},
    timeout=httpx.Timeout(connect=5.0, read=60.0, write=10.0, pool=5.0),
)


def enrich(adgangsadresse_id: str, kind: str = "ejendom", attempts: int = 5) -> dict | None:
    """Fetch /api/v1/enrich/adresser/{id}/{kind}. None if the source is still down."""
    for attempt in range(attempts):
        try:
            r = client.get(f"/api/v1/enrich/adresser/{adgangsadresse_id}/{kind}")
        except (httpx.TimeoutException, httpx.TransportError):
            r = None
        if r is not None and r.status_code not in RETRY_STATUS:
            r.raise_for_status()  # 4xx: fix the call, do not retry
            return r.json()
        retry_after = r.headers.get("Retry-After", "") if r is not None else ""
        wait = int(retry_after) if retry_after.isdigit() else min(60, 2 ** attempt)
        time.sleep(wait + random.random())
    return None


if __name__ == "__main__":
    data = enrich("0a3f507c-0624-32b8-e044-0003ba298018")
    print("No data right now" if data is None else sorted(data))

For long batch jobs it is better to put failing addresses in a queue and retry them later than to wait inside the loop. See Retries and backoff in production.

Next steps#

• Limits, quotas and 429
• Retries and backoff in production
• BBR building data