Implicit loading of d3 submodules?

It seems that there is some implicit loading of d3 core modules that are available to 3rd party libraries, but not to cells in the notebook. I had problems with https://beta.observablehq.com/d/9893a23d75042992 which I managed to pinpoint to being caused by d3-graphviz not seeing the same d3.transition as the notebook cells. This was solved in https://beta.observablehq.com/d/987eb142ded7be7d by not requiring the d3 bundle, but instead requiring the d3-transition sub module.

Some further debugging in https://beta.observablehq.com/d/545d9a40944beb47 showed that I didn’t need to require any d3 core modules at all to get d3-graphviz to work. The d3-graphviz I’m requiring is not a bundle, but somehow it gets the d3 modules it need anyway.

Can somebody explain why this happens?

1 Like

The only thing that’s implicitly loaded in an Observable notebook is the Observable standard library.

In addition, when using require, any dependencies of the module you require are also loaded. For example, if you look at the d3-graphviz bundle, you’ll see that it depends on a few other modules (viz.js/viz, d3-selection, d3-dispatch, d3-transition, etc.):

https://unpkg.com/d3-graphviz@2.1.0/build/d3-graphviz.js

These modules are loaded because they are dependencies of d3-graphviz, but they are not exposed on the object resolved by require because they are considered internal dependencies to d3-graphviz.

If you want access to these dependent modules, you need to require them separately. Note, however, that this won’t cause the scripts to be downloaded twice—require is smart enough to detect that it has already loaded a module and return the existing copy.

For example, you might say:

d3 = require("d3-graphviz", "d3-selection", "d3-dispatch", …)
Viz = require("viz.js/viz")

(Note that I’m using the “splat” capabilities of Observable’s require that allows combining of multiple modules into a single shared namespace object. This is nice for D3 modules since they are designed for this pattern.)

There are two unfortunate limitations here:

First, you can’t mix and match D3’s default bundle (d3) with individual D3 modules (such as d3-selection). Since d3-graphviz (correctly) depends on the D3 modules, your notebook must also load the D3 modules separately. We could fix that in D3 by providing an AMD-only bundle that depends on the D3 modules (as we already do for CommonJS). But for now you have to list the D3 modules by hand.

Second, Observable’s require doesn’t currently handle dependency version resolution. Meaning, when you require a module, it will always require the latest version of any dependencies, regardless of what’s declared in the dependencies block of the module’s package.json. You can specify versions at the top level, such as require("d3-selection@1"), but if you do this, it will no longer be considered the same module as the require("d3-selection") dependency of d3-graphviz, so then you’ll have two copies of d3-selection.

These limitations can be avoided by using ES dynamic import instead of require, since unpkg dynamically rewrites static imports to resolve dependency versions. However, the D3 default bundle also isn’t optimized for ES imports at the moment—it’ll trigger about 500+ requests for unminified code, making it much slower than require.

Our hope is that in the long term these challenges will evaporate with widespread adoption of ES modules and support for ES import. But we’re still a ways from that, so it may be needed for us to ship some interim improvements to require.

1 Like

Since I’m on my phone, just a brief thank you so much for the in-depth explanation and the bonus info. Using the latest version is no problem for me since I’m the author of d3-graphviz and I intend to keep it up-to-date with the d3 core modules.

1 Like

Good news, everyone! We just shipped a new release of d3-require (1.0! :tada:) that implements version resolution for dependencies, fixing the second limitation above.

1 Like