Help me avoiding runtime errors

Community Help
1

I’ve developed a tool which helps wikipedians compute gender statistics about the list of articles they’ve created.

The tools works fine for me: User-level gender statistics for Wikipedia / PAC | Observable

However, when users have created a lot of articles like SenseiAC (2,700 articles), they’ve got runtime errors : User-level gender statistics for Wikipedia / PAC | Observable.

Concretely, the notebook use Look at your list of created articles with the Xtools Page Prose API / PAC | Observable to get the list of pages created by a user. See here : User-level gender statistics for Wikipedia / PAC | Observable

I then call the Wikidata API for each article created here User-level gender statistics for Wikipedia / PAC | Observable using User-level gender statistics for Wikipedia / PAC | Observable.

This creates many API calls and therefore creates often runtime errors.

Do you know any strategy to avoid having runtime errors when you have many API calls?

2

You’ll probably want to throttle your API calls. A starting point would be to run them one after another:

From that you can create concurrency by running multiple queues in parallel. Things get slightly more tricky if you want to make sure that all queues will have roughly the same backlog size at all times.

2 Likes