D3 Charts

Show and Tell
1

We’re excited to announce a new generation of examples, D3 charts! :tada:

These new charts are structured as functions that take data and options. This design is intended to make it easier to reuse these examples out of the box: the charts have reasonable defaults, and can be configured through named options without needing to edit the code or fork the notebook. (Though you can still do that if needed!)

As functions, these new charts are also more amenable to reuse. You can import them into any Observable notebook (e.g., import {Histogram} from "@d3/histogram"). And you can copy-paste them into any JavaScript application (e.g., this React Histogram example on CodeSandbox).

We’ve ported about forty examples to this new style, and we’re working on porting other D3 examples over the coming months. We’d love your feedback. Thanks!

8 Likes
2

My two cents: I don’t quite get the focus on all of these out of the box charts. I always thought the value of working with D3 and Observable was learning how to build visualizations, understanding the core concepts and apply them to your situation. Using out of the box graphs is the antithesis of why I work with D3. Sometimes we have to wrap our visualizations as functions so they’re easier to employ in an application, so it’s helpful to see how you implement that. (For many years, I used your “Towards Reusable Charts” paradigm.) Very rarely in analysis work, client work, editorial work, large-scale application work, have I found an out of the box visualization able to quite suit my needs. To make it truly effective, I always have to do lots of customization with the data, whether thats with scales, labels, encoding methods, etc.

It’s funny because I have a number of notebooks that try to do something similar, but in practice, I almost always end up taking the original code and just editing it for my use case rather than come up with a lot of logic for handling more parameters. In some way, I’ve taken much comfort that this has been the case. Its nice to know that my work isn’t so easily automated. (Or perhaps, I’m just a crummy developer who can’t develop nimble charts.)

I’m sure there are a thousand good reasons for using this and Plot.js. In my experience, I haven’t seen a ton of value. They just feel too similar to the Excel/Google Spreadsheets/Tableau world that I ran so far from.

3

I understand your point but it’s not like anything is stopping you from developing fully custom built visualizations with D3 here on Observable. It’s just that D3 is only one of about a thousand libraries you might use on here. The fact that the team has built a large collection of out of the box tools on top of D3 only proves the value of D3 - if not the individual tools themselves. There are, though, plenty of folks with less experience using full-blown D3 who might appreciate the simpler (though less general) out of the box tools.

My two cents: +1 for Datasaurus if nothing else. :slight_smile:

1 Like
4

Oh wow! This is an amazing collection of precooked entrypoints for d3-based charts!

I’ve just gone through a couple of examples and I realized that D3 Charts do not follow the reusable chart pattern, but those are rather based on functions accepting data and config options, and returning the svg node. I’m just curious about the reason of this choice over the reusable pattern. Simplicity, mabye?

5

Even if you don’t use them as-is, I see few downsides to having the charts being easier to use out of the box. Like @mcmcclur said, you can still tinker with the code as you see fit either by copy-paste or forking.

You could maybe argue that these new charts are harder to tinker with since they do more than the old examples. But the additional complexity of these new charts is mostly useful complexity: they’re solving problems you were likely to run into when adapting the chart to your data set (e.g., handling label occlusion, or massaging tidy data into series). In the worst case you can delete the code you don’t need.

And I wouldn’t discount how convenient it is to have these as starting points for quick exploratory visualizations or prototypes, even if you end up customizing the code later. If I want a quick treemap, it’s great to have these functions available; I used import-with before, but these new functions are much more flexible and robust.

With Observable Plot, I find it remarkable how a high-level abstraction (compared to D3) means I see so many more visualizations than before—visualizations I wouldn’t have bothered trying to make with D3 because I prejudged them to be not worth the effort. So, even if you value D3’s expressiveness, I highly recommend having Plot in your proverbial toolbelt for exploration and prototyping.

7 Likes
6

Thanks, I appreciate hearing your thoughts. I’ll take your advice to try out more Observable Plot, but I’m still not sure about D3 Charts. I hope this feedback is helpful and not too nit-picky. I understand I can just not use it; I write this as someone who understands the struggle of learning charting libraries, both as student and a teacher.

TLDR; who is the audience for this? And why should they use D3 Charts vs. Templates vs. Observable Plot? Whats the right way to write D3 charts? As functions?

Let’s look at the D3 Charts’ Scatterplot. So if I wanted to make a scatterplot with D3 and I search for “Scatterplot” in Observable, this is the first notebook that will come up. If I were new to D3, I would likely think this is how I have to write it to make it work. I learned D3 by example, commenting out lines, changing datasets, so I wouldn’t have questioned the paradigm. (Maybe I would have also found D3 Scatterplot, and then I would be confused why there were two solutions.) I may not realize I don’t need to write it as a function and include so many parameters, which maybe to some is not that much more complexity? I think it might double the amount of code, which to a beginner, I’d argue, is adding a lot of complexity… And you mention that they are solving problems that I were likely to run into. Funny, the first problem I see with D3 Charts Scatterplot, the function doesn’t address…

In the Scatterplot example, there are labels that overlap, so I can’t read which car is which. As far as I can tell, there is no parameter that would allow me to adjust for this. I could make several charts that focused on specific ranges? A hover interaction/tooltip would be helpful. For that, I’m going to have to go under the hood and make some custom changes. So now I should fork your notebook and make my own variation with the hover interaction then import that notebook. The additional complexity of writing it as reusable function now seems lost on me. If I wanted to add a trendline, an ordinal color scale for car companies, etc., Observable Plot actually seems to be a lot more capable for this use case. I can implement the chart with little code and add tool tips and trendlines without getting under the hood.

Templates seems like the right solution for something out of the box that a beginner can use and be a gateway to learning D3. I could go to the data cell to just swap the data. D3 Charts seems to operate in a sorta non-Observable-like way. They don’t utilize the cells functionality. The entire function is written in one cell block, which I think for someone learning is harder to read than smaller snippets of code.

If you just want to make a chart quickly, then use Observable Plot, or whatever library you are accustomed to. If you want to make a chart with D3, you’re looking to make custom charts, right? Does D3 Charts help in that effort? It feels like a Betty Crocker-type solution, which I’d argue makes worse bakers. Just rip the band-aid off and learn what baking soda is. We’re not talking about making puff-pastry here. This is chocolate chip cookies.

7

We can add label occlusion to the Scatterplot implementation in the future (like we did for SlopeChart) and it will automatically propagate to imports of this function. Whereas if everyone copy-pastes code or forks the notebook, they have a snapshot that can only be updated by manually merging. The function serves as a helpful membrane (interface) that separates the implementation of the chart from its usage making it easier to update the implementation in the future.

So, yes, I do recommend that folks write their own charts this way, even if they’re planning on only using that chart once. It sets you up better in the future if you end up wanting to reuse that chart, even just as a starting point or prototype. And if you want to minimize the code, you can always start with fewer options and add them incrementally as you see fit. (That’s how we did it!) E.g.,

function LineChart(data, {x, y, width, height} = {}) {
  // your line chart here
}

They don’t utilize the cells functionality.

They do use dataflow, but at a higher granularity. For example, see the LineChart that exposes the hovered datum as a view (or generator). In practice, I don’t think there was a lot of value in breaking out all the pieces of a typical D3 implementation into separate cells (e.g., the scales, axes, dimensions, margins) because the inspector often didn’t reveal anything useful. And it served as a significant impediment to people that wanted to use the code outside of Observable, too.

2 Likes
8

A thousand times this. In my opinion people who followed this pattern often ended up with notebooks that were difficult to reason about, because you had to hunt for every reference to a cell instead of having clearly defined interfaces.

And on the topic of reusability: While you can import from another notebook repeatedly with different import overrides, I consider this to be crutch, not a feature. Organizing your code in distinct and clearly defined units of responsibility (i.e., functions or classes) is a skill that is fundamental outside of Observable, and should absolutely be considered best practice.

1 Like
9

Hi again folks, excuse me if this question might seem pedantic or slightly off topic, but I come from Python, hence JS design patterns are not my daily routine. I’m trying to wrap my head around what happens under the hood of this collection of new charts. AFAIU, D3 charts are designed this way:

 function TheChart(data, {<destructured chart options>} = {}) {

  // 1. Computing parts (e.g., values, axes, color scales, stuff...)

  // 2. DOM part (the viz)
  // const svg = d3.create("svg") ... 

  // 3. output
  return svg.node(); // or similar
}

So it means that you can import TheChart function in your notebook, call it passing in your data and your options, and then you get back your rendered chart… 🪄 automagically!

Now, what if I need to interact with the chart? For example: setting again just a specific option without creating a new chart from scratch, or getting back the value of an option already set. As I mentioned above, I used to believe that the reusable chart pattern is the way to go for this type of capabilities. And probably it is. I’d like to understand if the current design of D3 charts allows or not post-rendering interactions in some way. AFAIU, they don’t. Looks like there’s a different design choice here. In my poor pythonista eyes, D3 Charts seem mainly designed for a quick, reusable and tweakable entrypoint to the D3 approach. Am I wrong?

10

For the first, see the bar chart transitions example’s chart.update method:

For the second, see the multi-series line chart and index chart examples which emit input events on user interaction:

There are a variety of ways you can listen to those input events emitted by the chart. On Observable the simplest way is usually to declare a view, e.g.,

viewof focus = LineChart(data, options)

Then any cell that references focus will re-run whenever the user hovers over the chart. If you prefer to define them as separate cells you can also say:

chart = LineChart(data, options)

focus = Generators.input(chart)
1 Like
11

I agree breaking out every single component into individual cells is often not beneficial. Though putting everything into a single cell seems like a step too far in the other direction, especially in the future world where there’s 50 parameters.

12

I think this pattern is very exciting. I have a few questions about it.

Question 1 - Why is data so special?

With the constructor pattern, why is data given such special treatment as a separate argument?

In particular, why is it

function LineChart(data, {x, y, width, height} = {}) {

and not

function LineChart({data, x, y, width, height} = {}) {

?

Question 2 - How to emit various different kinds of events?

I see that the pattern that’s compatible with viewof is like this (from Line Chart, Index Chart):

svg
  .property("value", date)
  .dispatch("input", {bubbles: true}); // for viewof

First of all, why is {bubbles: true} included?

Secondly, is there any difference between these two

svg.property("value", date)

svg.node().value = date;

?

The main question is: how might might one adapt this pattern so that the chart can emit various different types of events, such as “markClicked” or “markHovered”? Would the following make sense:

svg.property("value", datum).dispatch("markClicked");

svg.property("value", datum).dispatch("markHovered");

?

I guess the question is, is it really a best practice to mutate node.value and expect the event listener to grab that synchronously like this

select(chart).on('markClicked', (event) => {
  const datum = event.target.value;
  console.log(datum);
});

?

Is there no alternative to this awkward mutation of the DOM value property that would allow us to place the value directly inside the event? Or, is it done this way because this is how it has been done in the past for native interactive DOM elements like input sliders? I suppose that makes sense, because seeing event.target.value is familiar to folks.

Question 3 - Any solid convention for the update function?

So far, the chart.update pattern seems pretty ad-hoc, as its arguments are totally distinct and disconnected from the arguments to the constructor. In particular I’m looking at the Bar Chart Transitions example.

In my view, the ideal pattern would allow invocation of chart.update with exactly the same set of options as the constructor, with the additional feature that if an option is omitted, it should fall back to what it was set to previously.

Anyway, lots of pros and cons to consider and evaluate, but this pattern looks most promising as a general replacement for the Towards Reusable Charts pattern! Thank you for all the hart work that has gone into this initiative.

1 Like
13

the functional approach means juice will work, so if you want to convert any option into a back-writable realtime updatable thing you can do it with

EDIT: ok made an example

https://observablehq.com/@tomlarkworthy/juice-and-charts

14

Because data is required, whereas options are optional. It’s similar to fetch(url, init).

Because it’s not the default behavior for events for whatever reason, but nearly every native event you encounter bubbles so it’s generally less surprising if your synthetic events also bubble. If you are listening to the SVG element directly, it doesn’t make a difference if your event doesn’t bubble, but if you’re listening to an ancestor of the SVG element, you’ll need the event to bubble to hear it.

Yes, if date is null selection.property will delete the property rather than assigning it to null. This mostly matters in the context of Observable where a view whose initial value is undefined will cause downstream cells to wait before running, whereas a view whose initial value is null will expose that null value immediately to downstream cells.

This is how native HTML inputs work. The inputs represent values that are controlled by the user, and emit an input event when the user interacts to change the value. The chart is “just” another input (albeit a fancy one). If you only expose the value on the event, there’s no way for the consumer to query the current value of the input outside of an event.

So, rather than exposing low-level events like “markClicked” and “markHovered”, you might consider having multiple inputs that you listen to separately. These inputs could be properties of the chart element using the EventTarget constructor and could represent the “clicked” state and the “hovered” state of the chart.

It’s ad hoc because implementing meaningful transitions from any state to any other state is practically impossible. For the transition to be meaningful you have to understand the semantics of what’s changing. Also in practice, you rarely need arbitrary transitions; more commonly, you need to handle something much more specific, like the domain of the scale changing (e.g., for zooming) or the values changing (e.g., for switching datasets or animating values over time). If you don’t like the generic name chart.update, you could always name it something more specific that conveys what you’re updating.

3 Likes
15

Thank you so much for your detailed reply.

Interesting. I’m struggling to imagine a use case where the consumer would want to query the current value outside of an event, at least for things like “markClicked” and “markHovered”.

Looking deeper into whether it’s possible to expose a value on an event, I discovered the “details” field of selection.dispatch. I suppose an alternative direction to the multiple inputs idea (which is really cool by the way) would be to only expose the details of the event in “details”, something like

svg.dispatch({detail: { eventType: 'markHovered', hoveredMark }});

I’m struggling to imagine a use case where one would instantiate an interactive chart in this way, and then add an event listener to an ancestor of it. When might that happen? What would be an advantage of adding a listener to an ancestor as opposed to adding it to the SVG directly?

Question 4 - Is it composeable?

If I may, I’d like to pose one more question on this pattern.

Let’s say you have a color legend “chart” that is implemented using this pattern. I’m curious how it would play out if one were to define a top level chart that invokes the color legend “chart” inside of it. Or, alternatively, a top level program could invoke both of these “charts” and assemble them into a single SVG. SVGs can nest, so I think it would turn out fine. Any thoughts on composeability? Thanks!

16

E.g. you want a chart(s) in a panel. I use bubbles extensively to create composite UIs. The main chapter of Scaling User Interface Development / Tom Larkworthy | Observable is dedicated to building UIs heirarchically, all of which uses bubbles to operate, it’s very useful, and is what enables composability.