Package integrity and yarn.lock/package-lock.json

Hi,

I’m embedding a notebook (Tissot’s indicatrix) as a dependency of my project. My package.json file contains:

  "dependencies": {
    "@fil/tissots-indicatrix": "https://api.observablehq.com/@fil/tissots-indicatrix.tgz?v=3"
  },

and I’m able to install with npm install.

But if I version the package-lock.json file (or the yarn.lock file), clone the repository on another machine and launch npm install, I get an integrity error, since the tissots-indicatrix.tgz file obtained on that machine from https://api.observablehq.com/@fil/tissots-indicatrix.tgz?v=3 is different (and the _integrity field in the tissots-indicatrix.tgz’s package.json is different indeed).

The same applies for yarn install / yarn.lock, or if I deploy to now.sh.

Obviously the solution is not to version package-lock.json nor yarn.lock, but I understand it’s a good practice to do so:

What do you recommend?

1 Like

I would recommend downloading the files and checking them into your source control, say like so:

curl https://api.observablehq.com/@fil/tissots-indicatrix.tgz?v=3 | tar xvz

@mbostock Shouldn’t package.json inside the tarball list all included ObservableHQ dependencies under bundledDependencies? Or is there a practical reason to avoid this?

OK, thanks, downloading the tgz seems to be the safest way indeed (moreover there is no indication of the history version in the URL, so any modification in the notebook or in one of its dependencies would obviously modify the package)

You can click “Download code” to view the latest revision id …

… which you can then insert into your tarball URL. Unfortunately this won’t affect any non-pinned dependencies inside the notebook.

I’m not familiar enough with either npm or yarn to know if there’s a custom resolver plugin API thingy that would allow to properly convert all dependency URLs, but if there was I’d imagine it could solve these kinds of problems.

I used the recommendation by @mbostock, it works perfectly. See https://github.com/severo/observable-to-standalone/blob/master/bundle/README.md on how to migrate a notebook to a standalone application.

4 Likes

may you please elaborate more on this, how it works? Because the command you propose itself odes not work (its not a gzipped package) and how we should require or import the JS files in our standalone application.

The structure of the tarball changed recently with the introduction of file attachments. I’ve updated my comment above. And for completeness, here’s a git repository that demonstrates the technique:

1 Like

Update: We’ve improved how we generate tarballs with respect to the modification times of the downloaded notebooks. So, if you yarn add a specific version of a notebook, it should no longer* fail the integrity check. For example:

yarn add 'https://api.observablehq.com/@fil/tissots-indicatrix@90.tgz?v=3'

Caveat: If you don’t pin a specific version, or if the downloaded notebook imports other notebooks without pinning specific versions, then the tarball may still change — but only because the notebooks changed, so that should be a useful signal.

3 Likes