Suitable rate limit for /document/{id}/meta

[mootari]

Hi,

I’d like to bulk-query the API meta data (https://api.observablehq.com/document/{id}/meta) for all of my notebooks. This route isn’t cached by Cloudflare, though. What is an appropriate request rate that won’t put a noticable strain on the service? 1 request / second? 5? 10?

Thanks!

[hellonearthis]

Wouldn’t once per day be sufficient? Or Once per hour.
The metadata can’t really be so time critical can it?

Documentation for api.observablehq.com

Hi @pentacular. Other than the API for downloading and embedding notebooks, we don’t yet have a public API. This may be something we offer in the future, and if we do, we will absolutely provide documentation for it. But in the meantime we do not recommend using our internal private API.

[mootari]

@hellonearthis

The metadata can’t really be so time critical can it?

The rate does not concern update intervals. The route targets individual notebooks, which means that a separate request has to be made for each notebook. In my case, this amounts to > 350 requests. The provided information is not available through listings like user/documents.

[hellonearthis]

Interesting, can you show an example of the metadata you’re collecting.

Made a calculator to see how gentle you can bang the server.
Doing 1 a minute would only take 6 hours.

Bang the server

[mootari]

I’m specifically interested in custom_thumbnail and forks. You can check the available data by opening any of your own notebooks notebook while being logged in and inspecting the request in your browser’s developer tools: