Coordinating "releases" across notebooks

I have a family of related notebooks that I’m starting to depend on externally. When I was in prototyping mode I just clicked “publish” or “reshare” each time things seemed stable.

Now that there are external things depending on these notebooks, I’d like to avoid externally-visible incompatibilities.

For simplicity, let’s say I have three notebooks (in practice I have more like 10): main, dep1 and dep2 , where main imports both dep1 and dep2, while dep1 just imports dep2.

Is there a (humane) way to keep both main and dep1 pinned to the same version of dep2?

Yes. Observable supports version pinning, both for npm libraries (require("d3@5")), as well as for notebooks.

Every time you make a change to a notebook, that version of the notebook is saved, and can be referred back to by version number. The easiest way to view all of the versions of your notebooks is in the History:

In this case, I’m looking at version 16, as you can see in the URL and in the header.

So if I want to pin to this version of the notebook:

import {check} from "28b643cedea3da3d@16"

… because it’s private or shared. If it’s public, that import will continue to work, but you can also write it like this:

import {check} from "@jashkenas/amazon-music@16"

That should allow you to keep main and dep1 pinned to a known working version, while continuing to work on dep2, and able to bump their imports at your leisure.

Thanks for the response, @jashkenas!

If I understand what you’ve outlined, my “release process” would require manually tracking and editing these version numbers from the leaves of the dependency graph up to the root notebook.

So, first find the version of dep2, then edit dep1 to include it. Now find the version of dep1 (that was created by setting the version of dep2) and update main to include that version of dep1 along with the initially chosen version of dep2. Am I understanding you properly?

This seems error-prone (and difficult to debug) even in this simplified case. If I’m understanding this correctly, it also makes testing at “HEAD” almost impossible.

Is it possible to somehow parameterize version numbers from the main notebook and pass these values down the import graph?

If so, perhaps we can use the Observable API to find the “latest” version of all imported notebooks and easily experiment with different values?

No need. You can use id@latest (or just leave off the version number, currently), to retrieve the latest version of a notebook. So:

import {check} from "28b643cedea3da3d@latest"

… in my case above.

Yes, pretty much. And I’m afraid I don’t see much of an alternative here. In order to precisely pin a dependency, you need for that dependency to be published at a known version which you want to pin.

That said, there’s quite a bit of flexibility here. You can test by importing from multiple versions at once, swapping out the different values from each:

import {check as check1} from "28b643cedea3da3d@10"
import {check as check2} from "28b643cedea3da3d@15"
import {check as checkLatest} from "28b643cedea3da3d@latest"

In the future, we’re hoping to support a style of version pinning for notebooks that pins by default, but allows you to explicitly bump the pinned version to the latest version when you want to upgrade — better supporting your described release process here, because you wouldn’t have to go look for the version numbers.

Interesting!

Things are becoming a bit clearer thanks to this discussion.

Since it appears that this is not valid syntax,

import {check as checkLatest} from `28b643cedea3da3d@${15}`

Is there a dynamic approach to importing notebooks that I could use to interpolate a (possibly overridden) value?

You can also use import … with to inject a specific version of dep2 into dep1.

1 Like

Oh, that might be the best option yet!

Though this doesn’t seem to work

import dep2 from "28b643cedea3da3d@latest"

Is there a different syntax that does essentially this?

If not, this implies that either all imported fields need to be fully enumerated or wrapped inside “top level” objects within dep1 and dep2.

1 Like

Yes, main would need to know which symbols from dep2 are imported by dep1, and then when main imports dep2, it would need to inject those same symbols from dep2 into dep1.

It’s all a little awkward, sorry, but hopefully it’ll be better when we implement automatic version pinning.

Got it.

I think I’ll end up wrapping everything that’s meant to be used outside a notebook in a top-level object.

That will make it easier to pass these values down to its own imports and the release process a bit easier to reason about.

If you want something like a “major” (backwards-compatibility-breaking) release of your notebook, and it is in wide use, consider making a fork, and linking to the new notebook from the top of the old one.

2 Likes

It would be nice if the @latest syntax mentioned by @jashkenas worked (currently I get ReferenceError: invalid notebook identifier; try a notebook name ("@observablehq/demo") or id ("5619eceafed5879d")). Leaving off the version number works for unpublished notebooks, but for published notebooks it only imports the latest published version.

I’m currently working on two previously-published notebooks in parallel and it’s a bit of a pain to have to keep bumping the version numbers in my imports.

1 Like

If you import by id, you’ll get the latest version that’s visible to you:

import {chart} from "09403b146bada149"

It’s only when you import by slug (that is, by the notebook’s public URL) that you’ll see the latest published version:

import {chart} from "@d3/bar-chart"

This is a bug on our side. The latter import should give you the latest version that’s visible to you, too; the behavior shouldn’t change based on whether you specify the notebook by id or slug.

I’ve filed a bug for us to fix. In the meantime, if you want to import the latest version of your notebook, you can import by id rather than by slug. You can find your notebook’s id by clicking “View history” in the notebook menu and then copying it out of the address bar.

2 Likes

I’m not totally sure I agree; it might be better to require an explicit opt-in for using a not-publicly-visible version. Otherwise it’s easy to publish a notebook that works for you but is broken for everyone else who can’t access unpublished versions of the dependencies.

3 Likes

I’m not sure which way it should go in the end, but the Introduction to Imports tutorial does seem to agree with you:

[…] Imports target the latest published version, so if I improve my map notebook, the new map will automatically appear here.

(The risk of importing the latest version is that your notebook may break if the imported notebook changes in a non-backwards-compatible way. […]

This would suggest that it’s the behavior of the import by id that’s bugged.