Most API breakage doesn't come from a request failing. It comes from a response changing shape without anyone saying so. A field that was always present becomes optional. A number becomes a string. A nested object gains a new required key, or loses an old one. Nothing errors at the network layer. Your code just starts receiving data it didn't expect, and the failure appears somewhere downstream (a crash, a wrong calculation, a form that stops validating), often far from the API call that caused it. That's API contract drift: the gap between what an endpoint used to return, what it returns now, and what your code still assumes.
This page is about detecting that drift. It doesn't cover full CI-enforced contract testing. If you own both sides of an API and want automated, consumer-driven contract verification in your deployment pipeline, Pact is the tool for that job. What follows is for a narrower and very common case: you consume an API you don't control, or you don't want to set up contract-testing infrastructure for it, and you want to know from real responses whether its shape has changed since you last looked.
Why these changes are easy to miss
A reshaped response still counts as a successful HTTP response. Unlike a 500 or a timeout, you get status 200, valid JSON, and no exception from the HTTP client. The break happens a layer up, in the code that reads a field that's gone or assumes a type that's changed. In dynamically typed code that tends to show up as a confusing runtime error far from the cause. In statically typed code with a hand-maintained interface, it can show up as nothing at all. The type checker checks your interface, not the live API, and a stale interface keeps agreeing with itself while the data changes underneath it.
Start with what the API says it returns
If the API publishes an OpenAPI or Swagger document, check that first. Specs are often wrong (more on that below), but a spec that isn't valid OpenAPI is its own problem and cheap to rule out. The OpenAPI / Swagger Validator checks a spec against the official schema for its version, and also flags $ref pointers that resolve to nothing and duplicate operationIds.
The guide to what OpenAPI validation checks explains where that stops. Validation confirms the document is well-formed. It doesn't confirm the live API behaves the way the document says. Those are separate claims, and treating them as one is a common way a "validated" contract still breaks consumers in production.
The gap between the spec and the response
A spec is a claim written by a person, and it's usually not regenerated from the server's behavior. It goes stale as soon as the implementation changes and nobody updates the document. That happens all the time: updating the spec is an extra step, easy to skip under deadline pressure, and nothing keeps the two in sync unless the team has built tooling to enforce it. A field can be added to the response without the spec changing. A field marked required in the document can be omitted by the server on some code path nobody tested against the spec.
The practical fix is to derive your working types from captured responses instead of treating the spec as the source of truth. That complements spec validation rather than replacing it. Spec validation asks whether the document is well-formed. This asks whether the document's shape matches what the server sent you, now, for the cases you hit.
Diffing two responses
The simplest way to catch drift is to save a response from an endpoint and later compare a fresh response from the same endpoint against it. The JSON Diff Viewer does this as a one-off structural comparison. It treats key order and array order differently: reordering an object's keys means nothing in JSON, while reordering an array's elements is a change. It also flags a field that changed type, such as a number arriving as a string. A plain text diff shows that as different text without making clear that the shape changed, and the shape is what broke your code.
That works well for a specific endpoint you already suspect has changed. It doesn't tell you which fields your code depends on and which are incidental. You still have to judge that.
A worked example
Say a /users/:id endpoint used to return this:
{
"id": 1042,
"name": "Ada Lovelace",
"phone": "+44 20 7946 0958"
}After a change on the provider's side, it returns this:
{
"id": "1042",
"name": "Ada Lovelace"
}Two things changed, and they're different kinds of change. id switched from a number to a string. It's still present and looks fine at a glance, but any code doing id === 1042 or arithmetic on it now breaks or misbehaves without throwing. phone is gone. Code that reads user.phone now gets undefined instead of a string, which may or may not throw depending on what that code does next.
Paste both responses into the JSON Diff Viewer and both changes show up plainly: a type change on id and phone marked as removed. Those are the two changes that matter, without the reformatting noise a line-based text diff might bury them in.
Now paste both samples, before and after, into JSON to Zod. Given only the "after" sample, the tool would infer id: z.string() and leave out phone. That schema is internally consistent, but it has erased the fact that id used to be numeric and that some responses might still send it that way during a rollout. Given both samples, multi-sample inference merges the variation instead of picking one: id becomes z.union([z.number(), z.string()]) and phone becomes z.string().optional(). The schema now records that these fields drifted, instead of reflecting whichever sample you pasted last. That's the advantage of multi-sample inference over single-sample codegen. The union and the optional field aren't guesses; they record variation observed across real responses.
Inferring types from samples instead of the spec
You could hand-write a type from the documentation, which may be wrong or outdated. You could base it on a single example response, which only shows what was present that one time: every field looks required, and a field that happened to be a string never becomes a union. Or you can infer the type from several real samples. JSON to TypeScript and JSON to Zod both do this. Paste multiple responses, and the output marks fields that are sometimes missing as optional and fields that sometimes change type as unions, because that variation appeared in the samples, not because someone guessed it. That's a stronger starting point than a hand-maintained interface or a single hard-coded example, since it comes from observed data.
The Live Types homepage tool adds one step. It runs the same multi-sample inference and lets you save the inferred shape as a snapshot, then diff fresh samples against it later. "Does this still match what I saw before?" becomes a check against a saved baseline instead of a manual re-comparison. It's still user-initiated: you paste fresh samples and run the diff. It isn't a background job that runs continuously. That was a deliberate scope decision. It keeps everything client-side with no server to host, secure, or trust, and the cost is that drift detection is something you run, not something that runs itself.
What this catches and what it misses
Sample-based drift detection tells you a response shape changed between two points in time when you captured it. It doesn't give you schema coverage across every response variant an API can produce. If a field only appears under a rare condition you never sampled, diffing two ordinary responses won't reveal it. It doesn't check business logic either: a field can keep the same type and hold a wrong value. And it doesn't run automatically unless you schedule it yourself, per the scope decision above. Keep those limits in mind so you don't treat it as continuous contract enforcement.
Three levels of checking
It's tempting to see this as a yes-or-no question: you have contract testing or you don't. It's closer to a range, trading setup cost for rigor. This site's tools sit at a specific point in that range on purpose.
At one end is no checking at all. You trust the spec, or trust that the API hasn't changed, until something breaks in production and someone traces it back to a changed response. It costs nothing to set up or maintain, and its failure mode is the silent downstream breakage this page started with.
In the middle is manual, sample-based checking, which is what this page has described. Capture real responses, diff them, infer types from multiple samples, and optionally save a snapshot and re-check later with Live Types. Setup cost is close to zero: no server, no CI wiring, no account. But nothing runs unless someone decides to run it, and it only covers the endpoints and cases you sampled, not a full contract.
At the other end is consumer-driven contract testing, which is Pact's territory. A consumer team writes its expectations of a provider's API as an executable contract. That contract runs against the provider's test suite in CI on every build and blocks any deploy that would break it. Setup is substantial, since both teams have to adopt the framework and wire it into their pipelines. In return you get a guarantee rather than a spot check: no build ships a break the recorded contract would catch, and nobody has to remember to run anything.
The middle option fits when both ends cost too much for what you need. Maybe you don't control the provider, so a shared CI contract isn't possible even if you wanted one. Or the integration is small or early enough that CI-enforced contract testing would be more infrastructure than what it protects is worth. In those cases the middle is a reasonable place to stay, not a stopgap.
Where this approach stops and Pact starts
Everything above is reactive, with a person in the loop. You capture samples, compare them, and decide what changed and whether it matters. None of it runs in a CI pipeline, blocks a deploy on a broken contract, or coordinates expectations between a consumer team and a provider team before either ships, and none of it was built to.
That's what consumer-driven contract testing tools like Pact are for. A provider runs a consumer's recorded expectations against its own test suite before deploying, so a break is caught before it reaches production, with both sides of the contract agreed and enforced as part of the build. If you're building that kind of cross-team, CI-gated guarantee, the Live Types vs Pact comparison lays out where the two approaches differ. They aren't competing to solve the same problem. They address neighboring problems in different situations, and working out which situation you're in is most of the decision.
Scripting this into CI without a shared contract
The manual workflow doesn't have to stay manual, and automating some of it doesn't require a shared, cross-team contract. The product roadmap includes an open-source CLI/GitHub Action that diffs a committed snapshot file against a freshly fetched sample during a normal CI run. It isn't shipped yet, so treat this as a direction, not a description of something that exists. It would do the same snapshot-diffing described above, triggered on a schedule or on every build instead of whenever someone remembers.
The difference from Pact still holds. That tool would run a one-sided, consumer-only check against whatever the provider returns at the time. It wouldn't be a two-sided contract that the provider's own test suite enforces before shipping a breaking change. It would automate the checking, not the coordination between teams. That's more convenient, but it isn't the guarantee a consumer-driven contract gives you.
A low-infrastructure workflow
- If there's a spec, validate it against the OpenAPI schema.
- Capture real responses instead of trusting what the spec says about their shape.
- When you suspect something changed, diff fresh responses against a saved baseline.
- Infer types from several real samples, not one example or a hand-maintained interface.
None of that needs a server, an account, or CI integration to start. It's quick to set up and manual by design. Once a team needs automated contract enforcement, it won't replace that.