The bottleneck is the rate limit, not throughput
The intuition is that bulk checking is about going fast. In practice a single lookup takes a second or two, so five hundred domains run strictly in series finish in under twenty minutes. That is not the problem. The constraint is how fast the upstream will let you ask.
Filing data ultimately traces back to the MIIT, usually through a cloud provider's lookup interface or a third-party database, and every layer has its own rate limit. We measured one public free interface whose demo key permits roughly one query every twenty seconds — five hundred domains would take almost three hours on that, and a single failure to back off triggers a run of rejections. The table below sets out the trade-offs.
| Approach | What you actually run into | Suits |
|---|---|---|
| Checking each one by hand | Workable but slow, and nobody really does it monthly | Under ten |
| Scraping the official lookup page | Captchas, layout changes and blocking; upkeep costs far more than expected | Not advisable |
| A free public interface | Tight rate limits, thin coverage of obscure domains, unreliable error reporting | Dozens, where misses are tolerable |
| A paid lookup API | Check the quota, the error semantics, and whether data is live or a snapshot | Hundreds to thousands |
| Your own cache plus a scheduler | Upstream limits still apply but can be spread out, and changes become detectable | Ongoing monitoring |
The second row is the one most people reach for first. Scraping the official lookup page looks like the cheapest option, but captchas, layout changes and blocking all become yours to handle, and every change on their side is a repair on yours — a cost that usually only gets counted honestly about three months in.
Three things that decide whether the batch finishes
These three separate a run that completes from one that stalls halfway.
- Back off by the number they give you When you are rate limited, many interfaces state in the error how long to wait. Retrying on a fixed interval keeps hitting the wall; reading that number and waiting exactly that long gets through on the next attempt. In our own testing a first run with a fixed one-second gap had fourteen of nineteen domains rate limited — reading the stated delay instead, the same batch completed in full.
- Cache the answer, never the failure Filing status changes rarely, so caching pays off. But a failed lookup must not be cached as a result. If an upstream timeout gets stored as "no filing," that error stays in your data, and because it is cached nothing will ever check again. Key the cache on whether the lookup succeeded, and retry failures instead of writing them.
- Store the change, not just the state Storing current state answers "how is it now." Storing each result with a timestamp answers "when did it go" — and that is the alert you actually need. A filing usually disappears weeks before anyone notices, and only a history lets you work out which change caused it.
Error semantics: where bulk checking goes wrong
Checking one domain by hand, you notice a response that looks odd. Running a batch, nobody is watching — so errors have to be structurally distinguishable, or they quietly become data.
This is not hypothetical. Testing one public interface, google.com came back as HTTP 200 with a code of 200, while icp, unit and the rest were all set to the string 查询失败 — "lookup failed." Code written as if resp["icp"]: mark_as_filed() records google.com as filed, with that error text stored as its filing number.
So before wiring in any source, confirm that three outcomes are distinguishable in the response: definitely filed, definitely not filed, and could not tell you this time. The third needs its own expression — a distinct status code or field, not a value in a data field. A source that cannot separate them should not feed anything that writes to a database. That is a separate question from accuracy, which we tested in full in our lookup tool comparison.
Build it or buy it
For a few dozen domains checked quarterly, a script against a free interface with back-off and retries is enough, and costs nothing. The implementation notes above are sufficient to make it reliable.
Paid options start to make sense in three situations: the count runs into the hundreds, you need continuous monitoring rather than a one-off inventory, or too high a share of obscure domains come back empty. That last one is easy to underestimate — free sources behave normally on well-known domains and can miss heavily on obscure ones, which are usually the domains you were asking about.
Three decisions, in order
Taking these in order avoids over-building:
1. Settle how often you need to check
A one-off inventory and continuous monitoring are different systems. An inventory finishes and needs no database; monitoring needs history, change detection and alerting. Decide which before deciding what to build.
2. Estimate how long a batch takes
Multiply domain count by the minimum interval per request. If that exceeds your scheduling window, adding machines will not help — that is an upstream rate limit, and the answers are caching, splitting the batch, or a different source.
3. Decide what happens when a lookup fails
The most commonly skipped design decision. How many retries, whether a persistent failure raises an alert, and above all never writing a failure as "no filing." Deciding up front is far cheaper than cleaning the data later.
Three things to confirm before wiring in any filing API
Ours or anyone's, these three decide whether it belongs in an automated pipeline:
- Rate limits, quotas and timeout behaviour The per-second and daily ceilings, what comes back when you exceed them, and whether an upstream timeout returns an error or "no filing." That last one is the easiest to meet only after going live.
- How failures are distinguished from "no filing" Look at the actual response structure, not the documentation's prose. Confirm errors get their own status code or field rather than being packed into the filing-number field.
- Whether the data is live or a snapshot This determines whether obscure domains resolve, and how long a lapsed filing takes to show. Ask for the snapshot's refresh interval and coverage.
In summary
The difficulty in bulk filing checks is rate limits and error semantics, not speed. The first decides whether a batch finishes; the second decides whether the resulting data can be trusted. Three implementation points carry most of the weight: back off by the delay the upstream states, never cache a failure as a result, and store changes rather than only current state.
At small scale, write it yourself — what is above is enough. If your count runs into the hundreds, or you need continuous monitoring with an alert when a filing disappears, write to us with your volume and use case and we will work out what fits.