my dataset has missing periods

Community Help
1

Hi,
I would like to ask for your help on the following point. I am working on a personal project that involves drawing a graph that shows data day by day. The data are measurements results with several different attributes. I am using timescale to have xScale that is able to show data that was measured on a given date.

xScale = d3.scaleTime()
    .domain(
      d3.timeDays(d3.timeDay(date), d3.timeDay.offset(date))
      )
    .range([margin.left, width - margin.right]);

Measurements are performed by an external system that saves results circa every minute, but with breaks. And these breaks are my problem. The code I have is drawing a line that connects last measurement before a break and the first one after a break. Could you suggest a solution that will not draw a line between these data points?

I am aware of d3.area.defined, however it seems it is not a solution. Line with Missing Data has data that have objects with date present and value as undefined. However, my dataset has no data for the breaks periods and d3.area.defined() is not a solution.

rgds Adam

2

Hi Adam. d3.scaleTime is a continuous scale, which means that the domain is typically specified by a start and an end value, rather than an array of discrete values. So, if you want a scale that spans one day in local time, you’d do something like this:

x = d3.scaleTime()
    .domain([d3.timeDay(date), d3.timeDay.offset(d3.timeDay(date))])
    .range([margin.left, width - margin.right])

As for the breaks, one approach you could consider is to draw discrete points rather than a line, say using a small circle or square for each point in your dataset. This would be a form of scatterplot, with x representing time and y representing the desired attribute.

If you want to draw a continuous line when you have data for the given minute, and gaps when you do not, then I’d recommend regularizing your data: binning and filling any empty bins with undefined or NaN values. Here’s how to do that using d3.group:

const valueByMinute = d3.rollup(data, ([d]) => d.value, d => +d3.timeMinute(d.time));
const regularData = d3.timeMinutes(start, end).map(time => ({time, value: valueByMinute.get(+time)}));

If there’s more than one value in the same minute, this will ignore subsequent values, but you can refine the logic as you like.

1 Like