Customer.io webhooks not working: the fixes that hold

On 15 October 1987, a few hours before the Great Storm broke, Michael Fish told BBC viewers that a woman had rung in about a hurricane. "Well, if you're watching, don't worry, there isn't!" (Wikipedia). He did go on to warn of high winds, and that morning the Surrey Mirror had warned of "furious gales". Overnight the storm brought gusts in excess of 100 mph to southern England. It caused around £1 billion of damage (Met Office). The Met Office counts 22 deaths; Wikipedia, citing the BBC, gives 19.

The warning was there: high winds in the same bulletin, furious gales in that morning's paper. What people remember is the opening line. The Met Office says the storm exposed gaps in forecasting and communication, and the three-tier National Severe Weather Warning Service followed the investigation into it.

Customer.io webhook debugging goes wrong the same way. The delivery log already holds the status code that explains the failure, and most teams open the workflow first.

TL;DR: Most Customer.io webhook failures are not Customer.io failures. They are auth headers, payload shape and receiving endpoints that answer too slowly, so debug from the delivery log outwards before you touch the workflow. A webhook action waits 16 seconds and retries timeouts, 408, 409, 429 and 5xx up to 11 times over approximately 1 hour. A 401 or 404 is not on that list, so fix it and retry by hand. A reporting webhook waits 4 seconds and retries for 7 days; the API reference says later events wait behind the failed one. Use Liquid's json filter in payloads, not escape, which changes the value rather than making it JSON-safe. Verify signatures against the raw body.

Why start at the delivery log, not the workflow?

The log tells you which side broke. Customer.io logs each webhook action request as a delivery with a status, the same as an email. Open Message activity, or the Sent tab of your automation or broadcast, and click the delivery. The detail page shows the reason for the failure (failed or attempted messages).

Three statuses do most of the diagnostic work (message statuses):

• Drafted: Customer.io generated the request and is waiting for you to send it. Webhook actions start in Draft by default, so nothing goes out until you switch them to Send automatically (webhook actions).
• Attempted: the hand-off is in progress and can mean a retry-able failure. Transient errors are retried with exponential backoff, up to 11 times over approximately 1 hour (failed or attempted messages).
• Failed: the request never left Customer.io. The docs say this is often a missing Liquid variable or broken Liquid logic.

Drafted is a setting. A failure with no response code is usually your template. Any response code points at the receiving end. Still choosing between the two? Read webhook actions vs reporting webhooks first.

The debug order: symptom, cause, fix

Work down this table.

Symptom in the log	Likely cause	Fix
Delivery shows Drafted, nothing sent	Sending behaviour still Draft	Switch to Send automatically, then send the queued drafts
Delivery Failed, never left Customer.io	Liquid could not render, often a missing attribute	Add a default fallback, click Fix, then Retry
401 or 403	Credentials missing, wrong or rotated	Correct the header or URL credentials, test, retry by hand
400 or 422	Broken JSON, wrong Content-Type or a missing field	Use the json filter, test against a capture URL
404 or 405	Wrong path or HTTP method	Check the URL and method; POST is the default
Timeout, 5xx, stuck at Attempted	Endpoint too slow or down	Acknowledge fast, process later
429	Your endpoint's rate limit	Rate limit the automation, or add a randomised delay
Connection refused, TLS errors	Firewall or certificate	Allowlist your region's IP addresses, fix the certificate chain
Your server rejects the signature	Parsed body or wrong key	Hash the raw body with the right key
Duplicate records downstream	Request accepted without a 2xx reply	Deduplicate on X-CIO-Idempotency-Key
Reporting events stop arriving	One failed call is holding the queue	Fix the endpoint before the 7-day window closes

What does Customer.io count as a failed webhook?

It depends on which kind, and the rules differ more than most people expect.

	Webhook action	Reporting webhook
Timeout	16 seconds	4 seconds
Retried on	Timeout, 408, 409, 429, 5xx	Timeout, plus a list including 400, 401, 403, 404 and 429
Retry window	Up to 11 times over approximately 1 hour	7 days, exponential backoff
After the window	Response attribute updates skipped, person moves on	Pending events expire, unrecoverable
Manual retry	Retry on the delivery or the Sent tab	None documented

Sources: webhook actions, reporting webhooks, API reference.

First, the webhook action docs name a retry only for a timeout and those four response types. A 400, 401 or 404 will not be retried for you. Correct the cause, then retry the delivery yourself.

Second, a manual retry is not a replay. Customer.io "sends a new request with the latest template content and attribute values" (webhook actions). Right after a template fix; wrong if the receiver needed the original values.

The two reporting pages also disagree. The integration page lists 422 among retried codes and the API reference leaves it out. The API reference says later calls wait behind a failed one, while the integration page describes batches that take new requests when there is room. Treat any non-2xx as a failure and plan for the stricter reading.

Is it the auth header?

A 401 or 403 means the receiving API turned your credentials away. That is never a Customer.io problem. Basic auth goes in the URL, as https://user:[email protected], and anything else goes in headers you can edit freely (webhook actions).

The usual cause is a rotated key. Someone regenerates a token in the CRM, and every automation holding a pasted copy starts returning 401. Since 14 January 2026 you can stop pasting. Reusable webhook configurations in Workspace Settings hold the URL, headers and auth once, and every automation that references them picks up a change (release note, 14 January 2026).

The configuration page carries three details worth knowing (reusable webhooks):

• Changes take effect immediately everywhere, so send a Test Webhook before you save.
• Header names and values support Liquid, including {{campaign.id}} and {{message.id}}.
• Team members with custom roles cannot create or configure reusable webhooks.

A 401 everywhere at once is a rotated credential; a 401 in one automation is its headers.

Is the payload shape wrong?

A 400 or 422 means the receiving API read your request and did not like it. In Customer.io the usual culprit is Liquid producing invalid JSON.

The webhook action docs warn against double quotation marks and line breaks inside attribute values, "because some services will reject the request" (webhook actions). A customer types a quotation mark into a free-text field, your template wraps it in quotes, and the JSON breaks for that one person. The log shows a 400 for some deliveries and a 200 for others, which looks random until you open the failing profile.

The same page suggests strip, strip_newlines, escape and normalize_whitespace to clean JSON. One of them needs care. The Liquid syntax list says escaping "removes special characters" and points to url_encode for URLs, yet its own example shows escape turning [email protected] into win%2Bhelp%40customer.io (Liquid syntax list). Whatever it does on your workspace, it changes the value rather than making it JSON-safe: the email address arrives altered.

The filter built for this job is json, which "Converts values to a JSON-encoded string". Drop the quotation marks around the tag and let the filter add them:

{
  "id": {{ customer.id | json }},
  "email": {{ customer.email | json }},
  "note": {{ customer.note | default: "" | json }}
}

This sample needs latest Liquid. On legacy Liquid, use to_json and an {% if %} fallback instead, as the failed-message guide shows. The default filter matters too. A missing Liquid variable can fail a message before it leaves Customer.io, and the failed-message guide's fix is a fallback such as default: "there" (failed or attempted messages). More traps of this kind are in Liquid anti-patterns.

Two more checks before you blame the receiver (webhook actions):

• Response size. The response is limited to 100 KB, and a larger one fails the webhook.
• Response mapping. If the Response tab sets attributes, the person waits until the update completes or every retry fails. If all retries fail, the update is skipped and the person moves on.

Payload trouble is often a data plan problem: if one event carries plan on Monday and plan_name on Tuesday, no filter saves it. A written event schema is the durable fix. We wrote a data plan when we moved AIRE Health onto Customer.io.

Is the receiving endpoint too slow?

Timeouts and 5xx responses are the receiving server's problem, and the budget is tighter than most developers assume. The Standard Webhooks specification recommends a timeout "somewhere between 15 and 30s" (Standard Webhooks). Customer.io's webhook actions sit near the bottom of that range at 16 seconds. Reporting webhooks give you 4 (reporting webhooks).

So accept the request, store it, return a 2xx, then do the work. Our boundary:

• Under 4 seconds on a bad day: handle reporting webhooks inline if you like.
• Between 4 and 16 seconds: fine for webhook actions, too slow for reporting webhooks. Queue the reporting events.
• Over 16 seconds, or unpredictable: queue both.

Retries create duplicates. Webhook action requests carry an X-CIO-Idempotency-Key. It repeats when your server accepts a request without returning a 20x, or when someone re-sends it from the UI (webhook actions). Store the key and ignore repeats.

What about rate limits?

On a 429, Customer.io looks at responses from the previous 10 minutes and works out a rate limit for you. You can also set one on the webhook channel of any automation, from 1 to 60,000 requests per minute, shared by every webhook action in it. There is no global limit (webhook actions).

People carry on through the automation while requests wait in the queue, unless you turn on Wait before continuing journey. Turn it on if later steps depend on the result. We cover what rate limits do further down a workflow in automation rate limits and skipped messages.

For bursty sends, a Random Delay block holds people for a random time you bound, to avoid overloading rate-limited API endpoints (randomised delay). Reporting webhooks have no equivalent: "No further rate limiting is available" (reporting webhooks).

Why does my signature check fail?

Every Customer.io webhook carries an X-CIO-Signature header: an HMAC-SHA256 of v0:<timestamp>:<body>, keyed with your signing secret, using the X-CIO-Timestamp header (webhook actions). When the check fails on a genuine request, it is one of two mistakes.

The first is hashing a parsed body. The docs say to use the raw body, not JSON.stringify() or json.dumps(). Capture the raw bytes before your framework parses them:

import hashlib
import hmac

def is_from_customerio(raw_body: bytes, timestamp: str, signature: str, secret: str) -> bool:
    message = b"v0:" + timestamp.encode() + b":" + raw_body
    expected = hmac.new(secret.encode(), message, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, signature)

The second is the wrong key. Since 17 May 2023, webhook actions have had a signing key separate from reporting webhooks (release note, 17 May 2023). The action key is under Settings > Workspace Settings > API & Webhook Credentials; the reporting key sits on the reporting webhooks page. Swap them and every signature fails.

The Standard Webhooks spec adds two habits worth copying: compare signatures in constant time, as hmac.compare_digest does, and reject timestamps outside an allowable tolerance to block replays.

Is a firewall blocking Customer.io?

If the delivery shows connection refused, check the firewall. Customer.io publishes the addresses its webhooks come from, by account region (IP addresses). By our count of the published table, the US region lists 17 addresses and the EU region 22. An EU workspace behind a US-only allowlist fails every request.

The page was last updated on 9 October 2026, so lists drift. For an automated check, the App API returns the current set from GET /v1/info/ip_addresses (List IP addresses).

TLS errors such as tls: failed to verify certificate: x509:, which the reporting docs list, point at your server's certificate chain, not at Customer.io.

Why have my reporting webhooks gone quiet?

No person waits on a reporting webhook, so it fails quietly. Four documented behaviours explain most silences.

• One failure can hold the queue. The API reference says Customer.io backlogs later calls until the failed one succeeds or the seven-day retry period expires.
• The integration can be marked invalid. The delivery flowchart ends one branch with "Your integration is marked invalid. Retry webhooks after an hour." (reporting webhooks).
• Seven days is a hard edge. Events not accepted within 7 days expire, and re-enabling a disabled webhook does not bring back what it missed (reporting webhooks).
• Strict schemas break on new fields. Customer.io may add payload fields without notice. Validate with additionalProperties: false and your endpoint rejects events the day one appears.

Two missing data complaints are configuration, not failure. Only the first click on a link is sent by default, unless you change Send Frequency. And a reporting webhook cannot be scoped to one automation; the docs suggest filtering on campaign_id at your end (reporting webhooks).

When is the workflow at fault?

The honest exception is a clean log because nothing was sent. No delivery at all? Check the action's conditions, because a person who does not meet them skips the step (why aren't people receiving my message). If the delivery says Drafted, it is the sending behaviour. Outside those cases, leave the workflow alone.

How do I test a webhook before it goes live?

The docs suggest a capture service such as webhook.site, test data only (webhook actions). For reporting webhooks they recommend a test or sandbox workspace, so you do not leak sensitive data (reporting webhooks). We explain how we run one in a Customer.io test workspace for staging.

Then use Send Test, which reports the status code and the payload you sent (Lob webhook guide). Test with awkward data: a quotation mark in a name, an empty attribute, a long free-text field.

For where webhooks sit alongside the APIs and reverse ETL, see our guide to integrating data sources with Customer.io. If a webhook keeps failing and you want a second pair of eyes, send us the details.

Frequently asked questions

How long does Customer.io wait for a webhook response?

Customer.io waits 16 seconds for a webhook action (webhook actions) and 4 seconds for a reporting webhook. If you need longer, return a 2xx once the request is stored (reporting webhooks).

How many times does Customer.io retry a failed webhook action?

Up to 11 times over approximately 1 hour, after a timeout or a 408, 409, 429 or 5xx response. A 401 is not on that list, so fix the cause and retry the delivery manually (webhook actions).

Where do I see why a Customer.io webhook failed?

Open the delivery from the Message activity page or the automation's Sent tab. The delivery detail page shows the reason and lets you retry (failed or attempted messages).

Why does my Customer.io webhook send invalid JSON?

Liquid is putting a quotation mark or line break inside a quoted string. Use the json filter on each value without surrounding quotes, and avoid escape, which changes the value rather than making it JSON-safe (Liquid syntax list).

Which IP addresses do Customer.io webhooks come from?

Customer.io publishes separate US and EU lists on its IP addresses page. Allow your region's list, or fetch it from GET /v1/info/ip_addresses (IP addresses).

Do Customer.io reporting webhooks replay missed events?

Not after the retry window. Deliveries are retried for up to 7 days, then pending events expire, and nothing is backfilled for a period when the webhook was disabled (reporting webhooks).

Why does my X-CIO-Signature check fail?

The usual causes are hashing a parsed body instead of the raw body, or using the wrong key. Webhook actions and reporting webhooks have had separate signing keys since 17 May 2023 (release note).

Sources

• Customer.io, Send and receive data with webhook actions, updated 14 September 2026.
• Customer.io, Reporting webhooks, updated 24 September 2026.
• Customer.io, Reporting webhooks API reference, read 12 October 2026.
• Customer.io, Configure reusable webhooks, updated 13 September 2026.
• Customer.io, Allowlist our IP addresses, updated 9 October 2026.
• Customer.io, List IP addresses, read 12 October 2026.
• Customer.io, Message statuses, updated 14 September 2026.
• Customer.io, Send failed or attempted messages, updated 14 September 2026.
• Customer.io, Why aren't people receiving my message?, updated 14 September 2026.
• Customer.io, Randomized delay, updated 14 September 2026.
• Customer.io, Liquid syntax list, updated 8 October 2026.
• Customer.io, Send direct mail with Lob, updated 14 September 2026.
• Customer.io, Securely verify requests for webhook actions, release note, 17 May 2023.
• Customer.io, Create reusable webhook configurations, release note, 14 January 2026.
• Standard Webhooks, Standard Webhooks specification, read 12 October 2026.
• Met Office, 30th anniversary of the 1987 storm, 11 October 2017.
• Wikipedia, Michael Fish, read 12 October 2026.
• NerveCentral, AIRE Health case study.