Promise.all too fast

1

Dear all,

I am downloading a remote data, but the host is rate limiting my download to a couple per second, is there a way to slow down the promise.all to lower rate per second?? different approach is fine too. thanks in advance

promises = tickersData.map(url => d3.csv(“https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol=“+url.Tick+”&apikey=demo&datatype=csv”) )

tickersdata= Promise.all(promises)

2

Dear @sunole,

Try this:

tickersdata = {
  var results = [];
  for (var i = 0; i < promises.length; i++) {
    await Promises.delay(1000);
    results.push(await promises[i]);
  }
  return results;
}

It waits for the promises to resolve, one at a time, with a second gap between each attempt. If you need to go slower, increase Promises.delay(...)

3

Another option is a library like p-queue that lets you control concurrency. Here’s a quick introduction:

1 Like