Best practices for non AMD/UMD package

Community Help
1

Dear All,

I’d like to try out Agentscript https://backspaces.github.io/as-core/ in a notebook, but it isn’t bundled using AMD or UMD. What’s the best practice for getting this to work; making a wrapper and putting it on npm, so I can pull it in with unpkg, making my own browser build on GitHub, or…?

Best
Simon

1 Like
2

Hi Simon,

I had the same questions when I tried to get clojurescript working in a notebook. I took the following steps:

  • find a similar library that will load in observablehq and see what’s different. I did this with mori and adapted it to work with clojurescript added. (have a look at how the file is being wrapped with umd code https://github.com/jeroenvandijk/mori/commit/51bd0d5f4d9ce007ae70d55e89068fc26eaff88f)
  • Test it locally, but with observablehq, by requiring a file from your local system. E.g serve it as a static file python -m SimpleHTTPServer 8000 and then use https://localtunnel.github.io/www/ to serve it from https (= required). You probably need to add a cachebuster to the url e.g. “?randomsomething” so require will reload the file.

When i was testing I noticed that my library gave the same error in a normal html file as with observablehq’s require. This problem was easier to understand (no observablehq in the middle). When I solved that problem it also loaded in observablehq actually.

So to answer your question, require can deal with (local) urls to files, use that first with a locally generated file and upload to npm when everything works.

HTH,
Jeroen

1 Like
3

as-core is an especially tricky one because it hasn’t been published to npm - this page should show it, but it looks like the author has chosen not to or just hasn’t yet published it. Which is a relatively rare situation - pretty much any JavaScript code that’s reused across organizations is published to npm.

The clearest options here are:

  • Forking the repo and publishing a version under your own handle, like @sdwfrost/as-core (but maintaining all authorship information, as to be a good open source citizen)
  • Working with the maintainer to get the setup fixed. It’s pretty clear from the readmes & site that the intent is to have dist files built by Rollup that others can import, but that clearly isn’t the current reality. So, it’s likely they’d appreciate some help making the project usable by others.
4

I managed to get something (nearly) working, as the minified as-core can be included inline, so that’s a start to try the various import approaches.

https://beta.observablehq.com/@sdwfrost/agentscript-core

I still need to flesh out the canvas to plot the x and y coordinates of the turtles, and debug the simulations.

1 Like
5

Nice! Never thought about copy-pasting a minified source :sunglasses: Well done!

6

There seems to be a character limit on cells, but fortunately, the package was little enough. I also have it working now (though it could be faster):

My notebook:
https://beta.observablehq.com/@sdwfrost/agentscript-core

Original simulation (using Three.js canvas):
http://backspaces.github.io/asx/models/?turtles

7

Hi. I’m the as-core author. For historic reasons I’ve never made a npm bundle. Thanks to Simon showing us Observable and getting us started, I’ll start making it more more, er, observable.

Here’s the plan:

  • Initially work on moving as-core to npm. I’ve never done it.
  • Figure out why UMD is preferred over es6 modules
  • Get hints on best practices.
  • Used scoped/org npm packages
  • Change repo names to something more sensible.

The group I work with, Redfish/SimTable (http://www.simtable.com/), would prefer “scoped” npm packages to avoid the name clash issues. But does that work with Observable’s require()? Indeed, the whole Observable import is a bit of a mystery. But Mike responded to a question that got me on the air, so to speak.

To give you a glimpse of noob’s issues: I’ve spent a lot of time being hyper modern: es6 modules, all repos having a github page for being a CDN, etc.

But then I find es6 modules are not first class in notebooks! I.e. es6 module import syntax doesn’t work, you have to use Observable primitives like “import” as a function.

And the CDN gh-pages really are inferior to npm. Hard to grok, given that the repos are mainly browser.

So in a bit, we’ll be on npm. First as-core, then asx. It’s kinda too bad Three.js and their examples/ fight, but thren
https://beta.observablehq.com/@tmcw/requiring-modules-troubleshooting
gave a lot of help. For asx, I’ll likely have to make some major changes.

Lastly, I’ve had several PRs to make my repos work on windows. I’m getting there but I was surprised just how big an issue that is in the node world. But, heck, node chops are a Good Thing … as is shelljs and shx.

8

Hi Owen!

Nice work on AgentScript, agent-based systems are super cool.

To cover a few of these questions:

  • scoped modules: Observable works well with scoped modules - luckily they’re pretty much the same as non-scoped modes expect for the fancy names. For instance, Mike’s notebook about Turf (a project I worked on and rallied for scoped modules) requires from @turf/turf@5, the scoped module path. Scoped modules are good.
  • import operator: This is very fresh, but it’s a part of JavaScript, rather than Observable-specific syntax - and, perhaps confusingly, is an operator instead of a function even though it looks like a function. It’s kind of like new in that way. Because it’s a JavaScript thing and it’s implemented in browsers, it’s another part of Observable that we don’t really mess with - your browser runs the import statement, no transpilation necessary. (pedantic note: import() is a stage-3 proposal to JavaScript, which means it isn’t on MDN docs or part of any spec, but is already being implemented by browsers and is near-guaranteed to eventally be adopted)
  • import syntax: Yeah, the static import syntax doesn’t work (yet), and that’s a bummer - we’re actively talking about how to weigh it against the cross-notebook import syntax, and hope to have a solution that makes import from work for ES modules.
  • Is UMD preferred?: we love UMD and ES modules equally - and I eagerly await the day when everything is ES - but ES modules still have a bunch of kinks to work out:
    • Importing libraries that are built with ES modules right now tends to get you the raw source code: like if I import my simple-statistics module, it’ll load all ~40 files in its src/ directory, in the browser. Which is a bummer for performance - even with HTTP2, loading lots and lots of files is still a drag.
    • Cross-dependencies are hard in both setups, but UMD is pretty clear about being able to require other modules as dependencies of a UMD bundle, whereas setting custom loaders with ES modules is still the wild west.
    • Basically it boils down to ‘JavaScript modules are still super diverse’ so here at Observable Inc we have some opinions about what’s best but also a prerogative to just make the ecosystem of code work roughly as-is, hence the long troubleshooting notebooks :slight_smile:
1 Like
9

Tom, I can’t thank you enough for the time you took to help here. I’m really grateful.

And yup, we’re going to have a redfish scope on npm, with agentscript as the first repo. And hopefully be “best practices”, especially for Observable-friendly.

I do have a last question here: Handling Dependencies for an es6 Module … basically how to manage dependencies within a notebook.

Note it doesn’t need to be for dependencies for es6 modules (as in the question above), just how do I get my code to be able to have dependencies within the notebook.

Thanks again!

1 Like
10

A npm package for agentscript is now available:

Here’s a notebook testing them:

11

@sdwfrost: could you test this in your notebooks? I haven’t converted asx yet (it will be @redfish/as-app3d). Thanks!

12

The require seems to work fine:

https://beta.observablehq.com/@sdwfrost/agentscript-core-v2

Thanks Owen!