Skip to content
← All writing

Signing webhooks, and the SSRF hole nobody patches

6 min read
.md
Cover illustration for Signing webhooks, and the SSRF hole nobody patches

TL;DR

Sign webhook payloads with HMAC and compare in constant time so a receiver can verify you sent them. Then guard the delivery side: a customer-controlled URL is an SSRF primitive, and it needs DNS resolution checked against private ranges before every request, including after each redirect.

My document processing platform posts a notification when a scan finishes. Customers paste in a URL and I send the result there.

Read that again. A stranger types an address into a form, and my servers go and fetch it. From inside my network.

I spent a week reading about webhook security before I noticed that. Every article was about HMAC signatures. They are all correct, and they are all about protecting the person receiving the webhook. Almost nobody writes about the person sending it.

That URL field is a way to make my infrastructure fetch anything a customer names. Including the things only my infrastructure can reach.

Signing, briefly

The receiver needs to know the payload came from you and has not been altered. Compute an HMAC over the raw body with a per-endpoint secret and send it as a header. RFC 2104 is the specification; Stripe’s webhook documentation is the clearest worked example of the surrounding conventions.

import { createHmac, timingSafeEqual } from 'node:crypto'

export function sign(rawBody: string, secret: string, timestamp: number) {
  const payload = `${timestamp}.${rawBody}`
  const mac = createHmac('sha256', secret).update(payload).digest('hex')
  return `t=${timestamp},v1=${mac}`
}

export function verify(rawBody: string, header: string, secret: string, toleranceSec = 300) {
  const parts = Object.fromEntries(header.split(',').map((p) => p.split('=')))
  const age = Math.abs(Date.now() / 1000 - Number(parts.t))
  if (!Number.isFinite(age) || age > toleranceSec) return false

  const expected = createHmac('sha256', secret).update(`${parts.t}.${rawBody}`).digest()
  const given = Buffer.from(parts.v1 ?? '', 'hex')

  return expected.length === given.length && timingSafeEqual(expected, given)
}

Three things there are easy to get wrong. Sign the raw body, before any JSON parse and re-stringify, because key order and whitespace change the bytes. Include the timestamp inside the signed payload and reject anything older than a few minutes, or a captured request can be replayed forever. And compare with timingSafeEqual, because === on strings returns early at the first differing byte and leaks the signature one character at a time.

MistakeWhat it costs youThe fix
Signing the parsed and re-stringified bodySignature fails at random, because key order and whitespace change the bytesSign the raw body, before any parse
Timestamp outside the signed payload, or absentA captured request replays foreverSign timestamp.body and reject anything older than 300 seconds
Comparing signatures with ===Returns early at the first differing byte, leaking the signature one character at a timetimingSafeEqual on equal-length buffers

Why is a customer’s URL a security problem?

Because your server is standing somewhere a stranger’s laptop cannot reach, and you just offered to fetch things on their behalf.

Try it from their side. Put http://169.254.169.254/latest/meta-data/ in the webhook field. On most cloud providers that address hands out the instance’s credentials to anything asking from inside the instance. Your delivery worker is inside the instance.

Same trick reaches your Redis, your Kubernetes API, an admin panel listening on 127.0.0.1, and anything else on the private network. The attacker does not even need to see the response directly. Your own delivery logs will show them what came back.

What a customer-supplied URL actually reaches:

TargetAddress a customer would enterWhat it hands over
Cloud instance metadatahttp://169.254.169.254/latest/meta-data/Instance credentials, on most providers
Loopback serviceshttp://127.0.0.1:6379/Redis, admin panels, anything bound to localhost
Private networkhttp://10.0.0.0/8, 192.168.0.0/16Internal APIs never meant to face the internet
Carrier-grade NAT range100.64.0.0/10Frequently forgotten by hand-written blocklists
A public hostname with a private A recordevil.example.comEverything above, past any string-matching filter

The OWASP SSRF prevention cheat sheet is the reference. The short version for a webhook sender:

  1. Accept only https. No http, no file, no gopher, no redis.
  2. Resolve the hostname yourself and check every returned address against the private and reserved ranges, rather than pattern-matching the hostname string.
  3. Reject on every redirect hop, re-resolving each new host, because a public hostname can 302 to 127.0.0.1.
  4. Cap redirects, response size and total time.
const BLOCKED_V4 = [
  /^0\./, /^10\./, /^127\./, /^169\.254\./, /^172\.(1[6-9]|2\d|3[01])\./,
  /^192\.168\./, /^100\.(6[4-9]|[7-9]\d|1[01]\d|12[0-7])\./,
]

export async function assertPublicHost(hostname: string) {
  const { address } = await lookup(hostname)      // resolve, do not trust the string
  if (address.includes(':')) throw new BlockedTarget('ipv6 not permitted')
  if (BLOCKED_V4.some((r) => r.test(address))) throw new BlockedTarget(address)
}

Resolving rather than string-matching is the part that gets skipped. Blocking the literal 127.0.0.1 in the URL does nothing against a hostname whose A record points there, and registering such a hostname takes about a minute.

There is a race here worth naming: you resolve, you check, then the HTTP client resolves again when it connects, and a DNS record with a one-second TTL can change between the two. Closing it properly means connecting to the address you validated and passing the hostname for TLS separately. Most teams accept the race. I mention it because pretending the check is airtight is worse than knowing where it leaks.

Retries, and the receiver’s side of the bargain

Delivery fails. Networks drop, receivers deploy, endpoints return 500 for ten minutes.

I retry three times with exponential backoff and then mark the delivery dead, keeping the attempt history so the customer can see what I sent and what came back. Three is low compared to the five or six many platforms use. It suits my failure profile: endpoints that are down for ten minutes recover inside three attempts, and endpoints that are down for a day are not coming back before someone notices.

Signing and SSRF protection are often discussed together and belong to different parties:

Receiver’s jobSender’s job
ThreatA forged or replayed payloadBeing used as a proxy into your own network
Primary controlVerify HMAC in constant timeResolve and validate the target address
Timestamp handlingReject anything older than the toleranceInclude the timestamp inside the signed payload
DuplicatesTreat the event ID as an idempotency keyGuarantee the ID is stable across retries
RedirectsNot applicableRe-resolve and re-check on every hop
What most guides coverThoroughlyAlmost not at all

Retries make duplicate delivery a certainty, not a possibility. A response can be lost after the receiver has committed. So every event carries a stable ID and the documentation tells integrators to treat it as an idempotency key, which is the same contract Stripe’s idempotent requests define in the opposite direction.

One operational detail that saves a support ticket every time: during a secret rotation, sign with both the old and the new secret for a period and send both signatures. Otherwise in-flight retries from before the rotation fail verification and look like an outage.

Sources

Common questions

How do you sign a webhook payload with HMAC?

Compute an HMAC-SHA256 over the raw request body concatenated with a timestamp, using a per-endpoint secret, and send it as a header. Sign the raw bytes before any JSON parse, because re-stringifying changes key order and whitespace and the signature will fail intermittently.

Why is a customer-supplied webhook URL an SSRF risk?

Your delivery worker runs inside your network, so offering to fetch an arbitrary URL turns your infrastructure into a proxy for anything only it can reach. Cloud instance metadata at 169.254.169.254 hands out credentials to anything asking from inside the instance, and your own delivery logs show the attacker what came back.

How do you prevent SSRF in a webhook sender?

Accept https only, resolve the hostname yourself and check every returned address against private and reserved ranges rather than pattern-matching the string, re-resolve and re-check on every redirect hop, and cap redirects, response size and total time. String-matching alone fails against a public hostname whose A record points at 127.0.0.1.

Why compare signatures with timingSafeEqual instead of ===?

String comparison returns as soon as it hits a differing byte, so response time varies with how many leading bytes matched. That leaks the expected signature one character at a time. A constant-time comparison over equal-length buffers takes the same time regardless of where the difference is.

How should webhook secret rotation be handled?

Sign with both the old and the new secret for a period and send both signatures. Otherwise retries that were already in flight before the rotation fail verification on arrival and present to the customer as an outage.

How many times should a webhook delivery be retried?

I retry three times with exponential backoff and then mark the delivery dead, keeping the attempt history. Three is lower than the five or six many platforms use, and it suits the failure profile: an endpoint down for ten minutes recovers inside three attempts, and one down for a day is not recovering before a human notices.

Written by Elson Tan, Head of Technology and co-founder at Nedex Group, working on AI harness and agent infrastructure.

AboutRSS
  • 6 min read

    Sandboxing code an agent wrote for you

    Once an agent writes and runs code, you are executing untrusted input on your infrastructure. The isolation question has known answers. The harder problems are quotas, cancellation and getting results out.

  • 17 min read

    The product was the easy part

    What a SaaS needs before it can charge anyone: credit billing in Stripe, invoicing and sales tax, email unsubscribe law, terms and privacy, and an admin panel.

  • 8 min read

    The job was never the code

    Most of my code is now written by an agent, and my output went up rather than down. That is not a story about typing speed. It is about what the job always was underneath the typing.

Get in touch

Tell me who you are and what you are working on.

Your details are used only to reply to this message.