Data not updating right?

Community Help
1

I have a notebook that fetches some data from an API, transforms it, and then charts it. Basic stuff, right?

Except it seems to get upset about the data sometimes, especially right after I change the URL or refresh the page.

To see the problem, first notice that their is an error on the data cell: “No data found in the last 183 days for mozilla-central”. In that cell, change the line that reads

const data = raw_data.filter(log => now - log.when < days * DAY);

to comment out the filter call:

const data = raw_data // .filter(log => now - log.when < days * DAY);

and re-execute the cell. The data should render correctly, ignoring the date range from the slider.

To me that seemed like my filter was wrong. I could live with that. That’s not the case though. If you un-comment that line, returning it to the original

const data = raw_data.filter(log => now - log.when < days * DAY);

then suddenly the data renders correctly, and the days slider works as expected. The same problem shows up, and the same steps fix it, when you change the “tree” cell to another value (examples: autoland or try).

This seems like a bug in data flow to me. What do you think?

edit: forgot to include the link to the notebook

2

Hi Mike!

Certainly an interesting one: here’s what’s going on: the line

  data.forEach(log => {
    log.when = new Date(log.when);
  });

Needs to be before

const data = raw_data.filter(log => now - log.when < days * DAY);

(and, it should be raw_data.forEach, rather than data.forEach)

Because log.when is a string for a date, rather than a date object, so it doesn’t cast to a number - it instead returns NaN, and since NaN is both > and < than all numbers, the filter will always return false, and you’ll get no entries.

When you comment and then un-comment the filter step, your code that runs the forEach on data is mutating the values in raw_data (because of normal JavaScript behavior) and thus when you uncomment that filter, you’re now using the mutated values - string dates parsed as Date objects, which work like you’d expect.

Hope that helps!

3

Ahhh, yes of course. That all makes sense now. Thanks for the catch!

I had originally had it the way you suggested, but flipped the filter around thinking it would be more performant to only transform the data that was in the range, without thinking too hard about the data types.