How to Watch an API for New Errors With curl and Run History

Plenty of APIs answer 200 OK and put the trouble in the body. A catalogue export says "status": "partial", and three of its 2,500 items carry an error object saying the price feed is stale. The status code says all is well, and the 720 kB of JSON is too much to read every time someone checks.

The response in this example is the catalogue export from our test fixtures, fetched with curl -si from a local server: 37,534 lines with the headers. The question is the one an on-call engineer or a coding agent has about any API it depends on. Is anything wrong in there, and is it new?

The same curl call run three times with a history key. Run 1 records one kind of error, run 2 is identical and comes back as one line, run 3 shows a new CURRENCY_MISSING error alongside the stale price errors that have been there for three runs.

What’s in the Response

curl -si https://api.example.com/v1/catalogue/export | abbreviatory reduce

[abbreviatory] HTTP response, 37,534 lines (719.6 kB), ref 358034d88165
http: 1 response: 200 OK; body 719.4 kB application/json
ERRORS (1 kind, 12 lines):
- x3 body /items/311/error: { "code": "PRICE_STALE", "message": "price feed for SKU-36051 is older than 24h" } [ref 358034d88165:g5]
OTHER (6 kinds, 37,522 lines):
- HTTP/1.0 200 OK [ref 358034d88165:g0]
- x3 (other headers) [ref 358034d88165:g1]
- Content-type: application/json [ref 358034d88165:g2]
- Content-Length: 719368 [ref 358034d88165:g3]
- (blank line) [ref 358034d88165:g4]
- body: JSON object with 6 keys, 30,020 values, 3 error fields [ref 358034d88165:g6]
    {
      "request_id": "4f9c2b1e-8d7a-4e21-9b3c-5a6d7e8f9012",
      "status": "partial",
      ...

The three stale prices are one entry, x3, because they differ only in the SKU number. Below them is the response in its own order: status, the headers that matter, then an outline of the body, with long arrays shown as their shape and a sample. The report is 1,754 bytes. A 4xx or 5xx status would have been an error too, and so would curl’s own complaints, like a refused connection.

Is It New?

Give the call a history key and run it on a schedule, from cron or a CI job:

curl -si https://api.example.com/v1/catalogue/export | abbreviatory reduce -key catalogue

The first run records the errors. The second, with the same errors, is one line:

[abbreviatory] HTTP response, 37,534 lines (719.6 kB), ref 358034d88165; run 2, identical to run 1: the same 1 error

To show a change, I edited one item in the saved response so it carried a different error. The third run leads with it:

run 3; since run 2: 1 new, 1 recurring (longest for 3 runs)
new: /items/1407/error: { "code": "CURRENCY_MISSING", "message": "no currency set for SKU-91210" } [ref d55f36188963:g6]

Each error field is tracked on its own, by its JSON path and its value, with array positions and numbers aside. So an error moving from item 311 to item 312 isn’t news, a new kind of error is, and fixing one of three errors on the same line shows as one gone. A redirect added in front of the response doesn’t change anything either; statuses are known by their code, not their place in the chain.

Under curl -v

curl -v works too. The request line and request headers, curl’s connection notes and the TLS chatter each fold into one group, and errors like * Failed to connect stay errors. A redirect chain reads as one: 301 Moved Permanently -> 200 OK.

Where It Stops

A JSON body is outlined when it parses; one longer than 64 kB on a single line, or bigger than 16 MB, is read like a log instead, and its references are by line rather than by JSON path. Bodies that aren’t JSON, HTML error pages for instance, are read like a log. A response body without a final newline that runs straight into the next response (curl -L with -i) runs into it. The reference has the rest.