Implicit loading of d3 submodules?

Community Help
1

It seems that there is some implicit loading of d3 core modules that are available to 3rd party libraries, but not to cells in the notebook. I had problems with https://beta.observablehq.com/d/9893a23d75042992 which I managed to pinpoint to being caused by d3-graphviz not seeing the same d3.transition as the notebook cells. This was solved in https://beta.observablehq.com/d/987eb142ded7be7d by not requiring the d3 bundle, but instead requiring the d3-transition sub module.

Some further debugging in https://beta.observablehq.com/d/545d9a40944beb47 showed that I didnā€™t need to require any d3 core modules at all to get d3-graphviz to work. The d3-graphviz Iā€™m requiring is not a bundle, but somehow it gets the d3 modules it need anyway.

Can somebody explain why this happens?

1 Like
2

The only thing thatā€™s implicitly loaded in an Observable notebook is the Observable standard library.

In addition, when using require, any dependencies of the module you require are also loaded. For example, if you look at the d3-graphviz bundle, youā€™ll see that it depends on a few other modules (viz.js/viz, d3-selection, d3-dispatch, d3-transition, etc.):

https://unpkg.com/d3-graphviz@2.1.0/build/d3-graphviz.js

These modules are loaded because they are dependencies of d3-graphviz, but they are not exposed on the object resolved by require because they are considered internal dependencies to d3-graphviz.

If you want access to these dependent modules, you need to require them separately. Note, however, that this wonā€™t cause the scripts to be downloaded twiceā€”require is smart enough to detect that it has already loaded a module and return the existing copy.

For example, you might say:

d3 = require("d3-graphviz", "d3-selection", "d3-dispatch", ā€¦)

Viz = require("viz.js/viz")

(Note that Iā€™m using the ā€œsplatā€ capabilities of Observableā€™s require that allows combining of multiple modules into a single shared namespace object. This is nice for D3 modules since they are designed for this pattern.)

There are two unfortunate limitations here:

First, you canā€™t mix and match D3ā€™s default bundle (d3) with individual D3 modules (such as d3-selection). Since d3-graphviz (correctly) depends on the D3 modules, your notebook must also load the D3 modules separately. We could fix that in D3 by providing an AMD-only bundle that depends on the D3 modules (as we already do for CommonJS). But for now you have to list the D3 modules by hand.

Second, Observableā€™s require doesnā€™t currently handle dependency version resolution. Meaning, when you require a module, it will always require the latest version of any dependencies, regardless of whatā€™s declared in the dependencies block of the moduleā€™s package.json. You can specify versions at the top level, such as require("d3-selection@1"), but if you do this, it will no longer be considered the same module as the require("d3-selection") dependency of d3-graphviz, so then youā€™ll have two copies of d3-selection.

These limitations can be avoided by using ES dynamic import instead of require, since unpkg dynamically rewrites static imports to resolve dependency versions. However, the D3 default bundle also isnā€™t optimized for ES imports at the momentā€”itā€™ll trigger about 500+ requests for unminified code, making it much slower than require.

Our hope is that in the long term these challenges will evaporate with widespread adoption of ES modules and support for ES import. But weā€™re still a ways from that, so it may be needed for us to ship some interim improvements to require.

1 Like
3

Since Iā€™m on my phone, just a brief thank you so much for the in-depth explanation and the bonus info. Using the latest version is no problem for me since Iā€™m the author of d3-graphviz and I intend to keep it up-to-date with the d3 core modules.

1 Like
4

Good news, everyone! We just shipped a new release of d3-require (1.0! :tada:) that implements version resolution for dependencies, fixing the second limitation above.

1 Like