In defense of static imports

[mourner]

From Observable’s Not JavaScript:

Since everything in Observable is inherently dynamic, there’s not really a need for static ES imports—though, we might add support in the future.

I’d like to disagree with this. More and more open source libraries are published as ES-Module-only, meaning you can’t use require, and it doesn’t feel good to use it in a modern ES environment like Observable. Dynamic imports on the other hand feel quite awkward and inconvenient compared to static imports syntax in several ways:

a) If the library exposes itself through a default export (which is the case for a lot of libraries, if not the majority), you’re forced to write this construct that’s unfriendly and unexpected to most users:

Delatin = (await import('https://unpkg.com/delatin')).default

b) With static imports of Observable cells, you can import multiple utility functions at once:

import {slider, button} from '@jashkenas/inputs'

With dynamic imports, you’re forced to namespace them:

inputs = import('https://unpkg.com/foobar')

c) Inconsistency. It would be easier for beginners if they didn’t have to remember that you can only use static imports for cells, and dynamic imports for external modules, and this choice contradicts with this argument:

Since everything in Observable is inherently dynamic, there’s not really a need for static ES imports

If everything in Observable is inherently dynamic and there’s no need for static imports, why are they used for importing cells? I understand that there was a need to differentiate between resolving non-URL names to Observable vs NPM, but using static vs dynamic imports for this doesn’t feel right.

Technically, you could expand static imports syntax so that it resolves URLs to external modules and non-URLs to Observable notebooks. Limiting NPM name resolution to “legacy” require feels right — see also Pika Web and Deno for examples where explicit resolution was chosen as preferred, inspired by browser ESM implementation.

It’s a subtle thing and not a deal breaker for me, but thought I’d my two cents because I think static imports of external modules would be a really awesome improvement!

[mbostock]

I agree that the syntax of static imports is preferable, and that it would be nice to support static imports for ES modules (as well to continue to support notebook imports, obviously).

This doesn’t directly address the issue, but we are hoping to add support for cell destructuring in the near future. This makes the dynamic import syntax less cumbersome in your example (note that the await is implicit because Observable automatically awaits promises at cell boundaries):

{default: Delatin} = import("delatin")

Which would be equivalent to this static import syntax (outside of Observable):

import Delatin from "delatin"

Putting that aside, the problem currently is that we need some way to disambiguate between imports of notebooks and imports of ES modules. For example, if you say

import {legend} from "@d3/color-legend"

do you mean the legend cell from the notebook @d3/color-legend or the npm package of the same name? The latter doesn’t currently exist, but we don’t want a shared namespace between Observable notebook identifiers and npm package identifiers. So we use static vs. dynamic imports to disambiguate imports.

What the alternatives we should consider? We’d like to allow static imports of ES modules, but the simple approach would break compatibility with existing notebooks, and we’d like to avoid versioning the language (if at all possible). We could do something like

import Delatin from "es:delatin"

where the “es:” prefix indicates an ES module (as opposed to an Observable notebook), but that’s nonstandard, and especially weird when it’s a URL

import Delatin from "es:https://cdn.jsdelivr.net/npm/delatin"

Related, if we want to go all-in on ES imports, we’re going to need browser support for import maps and possibly some Observable support for configuring import maps. The challenge with ES imports currently is that all (nested) imports must be relative or absolute paths, not bare identifiers. This makes it difficult to use cross-package imports, such as d3-scale importing d3-interpolate. We currently use unpkg’s “very experimental” support for rewriting bare import specifiers, but that won’t be compatible with Observable’s future support for versioning pinning since unpkg currently rewrites the bare specifiers to the latest published package that satisfies the semver range in the associated package.json. require does not have this issue because the require function does the resolution, giving us a hook to control the behavior at runtime.

I hope this clarifies some of the challenges with your request. We’re all ears! Eager to hear your ideas.

[mourner]

Mike, thanks for the thoughtful response!

Regarding disambiguation between NPM and Observable cells — I covered this at the end of the post, but let me clarify: I think Observable should NOT support NPM resolution with static import syntax at all:

import Delatin from 'delatin' /* ERROR — URL or Observable ID expected */
import Delatin from 'https://unpkg.com/delatin' /* External import */
import {legend} from '@d3/color-legend' /* All non-URLs are treated as cell imports */

This would get rid of all disambiguation without the need for any prefixes. It also doesn’t break compatibility because you only add support for something that currently errors (static import from a url) without changing any other behavior (require, dynamic import etc.).

You can also draw a parallel between Node and Observable:

• In Node, if require is given a relative or absolute path, it resolves to it; otherwise it looks in node_modules
• In Observable, if static import is given an URL, it would resolve to it; otherwise it would look among Observable cells

[mbostock]

Unfortunately that does not avoid the need for disambiguation because you can import Observable notebooks by URL also:

import {chart} from "https://api.observablehq.com/@d3/bar-chart.js?v=3"

It’s admittedly a rare case where someone, say, publishes a compiled Observable notebook to npm. But it is supported.

(Though if we know the full URL, we could determine whether the imported module is an Observable notebook rather than a vanilla ES Module but looking for a special export, similar to the __esModule export convention. But we couldn’t determine that statically just by looking at the code.)

[mourner]

Are there any known cases of someone importing a compiled Observable notebook by URL rather than directly? You could run a query in your database for such cases, and if none are found, drop support for it now to open up the possibility of adding static imports for modules.

[bgchen]

Although I just learned about the “import Observable notebooks by URL” feature, I want to vote against dropping support for this completely (although arbitrary syntax changes would be fine) since I can see myself wanting to import compiled notebooks from e.g. github in the future.

[Fil]

Is this related? the hello delatin notebook suddenly broke with this error:

(Solved in ☠️ Bug: Dynamic imports are broken ☠️)

[mootari]

@Fil

the hello delatin notebook suddenly broke with this error:

I’m seeing the error in other notebooks that use the same construct (e.g. Orientation / Fabian Iwand | Observable).

[mourner]

Although I just learned about the “import Observable notebooks by URL” feature, I want to vote against dropping support for this completely (although arbitrary syntax changes would be fine) since I can see myself wanting to import compiled notebooks from e.g. github in the future.

@bgchen but if there’s a choice between supporting Observable notebook import by URL and static imports of external modules by URL, would you still choose the first option? I’d guess that the latter would be useful for many orders of magnitude more people.

[bgchen]

You’re right, but I don’t see why it has to be only one or the other (dammit I want both! ). @mootari’s comments here made sense to me:

Feature request: conditional imports

If I might add my two cents: I think the current import with syntax should be deprecated and replaced with an import helper in the stdlib or runtime that works similarly to d3-require (and would likely just end up being a custom resolver?). Let’s be honest, the current syntax is confusing as hell when you’re not familiar with this Observable feature: syntax mimics static import syntax, but only supports a fraction of the actual syntax allows only notebook imports notebook imports look like np…

Which means I guess I’d be OK with a new version of the “Observable language”. As I said I’m not too fussy about syntax, as long as the capability is still around.

[rreusser]

Import maps seem very interesting! People seem to like the idea (" Consensus: Web Developers:
Strongly positive") but it’s not quite clear to me how they’d get distributed, like as part of the module so that it can resolve its dependencies, sort of like how tools look for types/, or out of line via notebook/tool configuration.

FWIW I’m currently working through an ES module distribution of stdlib. You can see an example notebook here. It currently works, and a final product feels tantalizingly close, but I’ve been somewhat surprised just how challenging the last few percent has been. We chose to create a single repo and rewrite paths as relative since out of a large repo (94MB unpacked, which is almost all just a couple sample datasets you’d almost never need to load) you can easily import just a few kilobytes of a function and its dependencies. But the fancy CDNs (e.g. snowpack.dev, deno.land/x) really seem to struggle to fetch or minify it or something, perhaps due to the size, and the simple CDNs (e.g. unpkg) seem to struggle with caching and the ?module transform, causing timeouts and failures for the first few tries, so that I feel like publishing it might actually need to include a script to hit all the entry points and trigger fetching/caching.

(If I’m being honest, I’m a little disappointed the size of the repo seems to be an issue since I thought that was the whole point of ES module distributions! )

An import map would increase complexity of building, but might avoid some of these issues by allowing to split into smaller modules à la d3—but then it might also be compensating for issues that just need to be fixed at the CDN level.

FWIW seeing deno work effortlessly with ES modules from github for the first time was kinda stunning

import { besselj0, ellipk } from 'https://github.com/stdlib-js/esm/raw/main/math/base/special.js'

console.log( besselj0( 1.0 ), ellipk( 0.5 ) );
// => 0.7651976865579666 1.854074677301372

Sorry this is kind of meandering, but I’d love to know if people have made progress on tools and design patterns at the confluence of ES modules and Observable. It all seems like the future, but I’m still not quite sure how much of this is things I don’t yet know vs. patterns and tools which are still being developed.