how to move from d3.group to a format that works for a sunburst chart?

This is a follow-up to my question on Rendering Mapped / Grouped Data.


EDIT : I originally thought this question about about recursive renaming of object keys and values, but re-considering it, my question is essentially this: How do I move from data organized by d3.group into a format that I can plug into a Zoomable Sunburst chart?


Following @mootari’s advice, I did some reading into Array, Set and Map methods. More specifically, I learned what a method really is, and got a crash course on JS fundamentals, which has been very helpful. Thanks Fabian!

However, while I am now more capable in manipulating data, I still haven’t managed to figure out how this would be done recursively - such as when trying to coax grouped data into a format that expected for the Zoomable Sunburst chart.

For example - I pulled some data on projects financed by the Asian Development Bank, and then used d3.group to organize these data first by ‘status’, then ‘country’, then ‘sector’. The code is this:

grouped_data = d3.group(sovereign_projects.slice(1), d => d["Status"], d => d["Country"],  d => d["Sector"])

and the output looks like this:

If I am not mistaken about these terms (sorry, still learning), the map ‘key’ becomes the value for each of these elements (e.g. for ‘status’, one key is ‘active’), and then the value assigned to that key becomes the corresponding ‘map’, which is an array of objects.

Answering my question on Rendering Mapped / Grouped Data, @mootari showed me how to create a new array comprised of the key name and the size of the corresponding map:

grouped_data_array = Array.from(grouped_data, ([key, map]) => ({
  name: key.toLowerCase(),
  value: map.size,
}))

This works great for a normal pie chart, but what if I want to ‘drill down’ deeper into each level?

To plug this into Mike’s Zoomable Sunburst chart, I’d need to perform this re-working of keys and values recursively, so that it matches up with the way the chart is expecting data (or, i think i need to do this?):

From what I can gather, part of this is accomplished with d3.hierarchy, but beyond that I haven’t figured out how I to accomplish a proper transformation (although I’ve been reading and tinkering for a few weeks now).

Would you be willing to help me figure this out?

Alternatively, would you be willing to tell me this approach is folly?

For some additional context:

I’m really guessing that it’d be possible, or relatively easy, to re-work the data rather than to tinker with the visualization code based on the idea that a visualization can be ‘reusable’. In this case, I was hoping to do something like this:

import {chart}
with {grouped_data as data}
from "@d3/zoomable-sunburst"

But is this really the way to go? Or should I instead be editing the visualization code itself to match the grouped data (rather than trying to make the grouped data match the expected inputs for the visualization)?

I would really appreciate your help and insights! :pray:

Here’s a notebook with the data I’ve been playing with:

Thanks in advance, community! You’re all swell folks, and I appreciate learning from you!

Related Readings:

1 Like

Hey Aaron, you might find this useful https://observablehq.com/@bayre/unrolling-a-d3-rollup#nest. It’s a function that converts a d3.rollup into a nested object, the data structure that is used in most d3.hierarchy examples.

With D3 v6 you can create a hierarchy with d3.group/d3.rollup directly, but it requires the chart to be restructured a little. This diff shows the edits I made to the Treemap to get it running on d3.rollup https://observablehq.com/compare/a1fd3857bac219b0@164...1b6b102e10f3f36d@190.

3 Likes

Hi @bayre, and thank you!

So from the look of the resulting data, your nest function does the trick:

… And yet when I try plugging this into the chart, it’s still coming up blank.

Any insights into what I’m missing?

EDIT: Sorry, I caught it - I just had to specify that I wanted the length of the final ‘value’ tier.

What a neat function! Thank you! :smile:

You can compute a d3.hierarchy directly from d3.rollup:

rollup = d3.rollup(sovereign_projects.slice(1), d => d.length, d => d["Status"], d => d["Country"],  d => d["Sector"])
root = d3.hierarchy(rollup)

The problem is that D3 Zoomable Sunburst example uses d => d.value as the value accessor for computing the value of each node. If you use d3.rollup as input, then the leaf nodes are [key, value] entries, and so you need to say instead:

root = d3.hierarchy(rollup).sum(([, value]) => value)

There are a few other changes that need to be adapted from that example. Here’s a working fork with your data:

And here’s what I changed:

4 Likes

Thank you, @mbostock . This is helpful!

Just to make sure that I understand all of what’s going on:

I was using D3.group to help me drill down into my dataset according a preferred pathway (in my case, ‘status’, ‘country’, ‘sector’). d3.gourp is particularly helpful in the sense that it the data are re-arranged, but otherwise intact (that is, allowing me to read through the groupings with all data attributes and values form the original data set being visible and accessible–even beyond the final grouping tier.

The solutions here, however, both rely on d3.rollup. Reading @Fil’s documentation:

d3.rollup lets you “reduce” each group by computing a corresponding summary value, such as a sum or count. The result is a map from key to reduced value.

From this, my expectation is that switching from d3.group to d3.rollup would return the data in a similar structure, but with values from the groupings rather than full, re-grouped arrays. I also expect/imagine that this is necessary as I move from wishing to ‘explore’ the data to wishing to ‘render/visualize’ the data.

But as I try to break this down to understand it, matters become very confusing, i.e. -

When I do a simple substitution, I am returned a very different looking map:

grouped_data = d3.group(sovereign_projects.slice(1), d => d["Status"], d => d["Country"],  d => d["Sector"])

vs.

rolled_data = d3.rollup(sovereign_projects.slice(1), d => d["Status"], d => d["Country"],  d => d["Sector"])

In your response (and now I see, too, in Fil’s example), you added a first mapping: d => d.length, which returns a similar structure as with d3.group:

Drilling into the data, I see that adding the length mapping gives me actual number values at the final tier of my grouping rather than returning undefined, and I conceptually understand why this would be the case (otherwise I haven’t instructed what to do with at the end of the mapping), but I don’t quite follow why the results without specifying length would result is such a different in the map structure itself: in my case, why it’s returning ‘country’ groupings and totally skipping ‘status’ groupings until length is specified.

I suspect this relates to another statement in Fil’s notebook:

The reducer is only applied to the leaf groups: above, counting the athletes for a particular nation and sport. The reducer is not applied to the parent groups, or to the values as a whole.

Does this imply that the length step is necessary to establish a ‘parent’ group, so that reduction is performed on my desired first step ‘status’?

And if I am working through d3.group to explore my data, must I still switch over to d3.rollup to visualize it, or can I stay with d3.group without causing myself a lot of headaches?

I really appreciate everyone’s time and help!! Thank you!

The second argument to d3.rollup is unlike the others — it is a function that takes a group and returns a value for that group. You can recover the functionality of d3.group from d3.rollup by passing in a function that just returns the group itself as the “rolled-up” value:

d3.group(xs, f1, f2, f3)

will compute the same result as

d3.rollup(xs, g => g, f1, f2, f3)

Passing in a different function, such as g => g.length, will use that to compute the final value. In your case, the d => d["Status"] function was being passed the group as the d argument, which is why the result was undefined.

1 Like

Thanks @yurivish!

Out of curiosity, how did you know this? Looking at the d3.rollup documentation, I see the pattern you explained here, but not the explanation :wink:

The second argument is in the documented signature as reduce and I’m familiar with the idea of a reduction from functional programming (e.g. MapReduce).

The docs could probably be improved to mention that for each group, the reduction function is passed an array of the values in that group and the return value is used as the value for that group.

1 Like

I’ve added a paragraph at https://observablehq.com/@d3/d3-group#rollups to try and clarify this point. Let me know if it can be improved.

2 Likes

Thanks @Fil!