🏠 back to Observable

Export final code?

This would really awesome. Talking use cases, I think it would be benificial to enable at least two kinds of exports: not just (1) a javascript bundle with all the newest javascript bundling details that modern web developers are into, for coders, but also (2) a ‘immediately usable blog-post’ kind of export, say with a html file included that layouts a document somewhat like the original notebook (maybe including reacitivity, maybe without, I’m not sure). This would target not-yet-so-experienced javascript developers, teachers, visual coders not into the details of javascript packaging, etc.

1 Like

Thank you for all of your great work!!!
I found Observable when I was looking for a solution to integrate d3.js into tiddlywiki5 for its comfortable UI and standalone notebook style.
So, question for the Wizards: is it a totally crazy idea trying to integrate the two system? I mean Observable and tiddlywiki5? Would their architecture allow trying it?
Yours sincerely, CodeXmonk

I like observables for really allowing me to understanding what is happening with the data and with concept visualization, but at some point I need to get back to writing Javascript for specific Web Applications. I know from Tom MacWright that the plan is to be able to export an observable as pure Javascript. Any thoughts from the community? Is this something you really want, or is it not very important (Observables are fine on their own)

I would really like to be able to export the awesome Observables into Javascript so that I can work with either platform. I think it would strengthen the importance of the Observable as a Javascript teaching tool. Presently it is easy to take a Javascript page and make it into an observable, but the other direction is fairly difficult to do.

1 Like

I think exporting will be great feature because

  1. Many Independent observable notebooks, will be very good static standalone web apps themselves (living on separate domain)
  2. Notebooks can be injected into separate things (web apps, blogs, articles )
  3. We may want to hide some variables info (a.k.a, edit generated code)

    I think this list will be big
4 Likes

I am loving Observable for my own data analysis and exploration. Its extremely easy and flexible.

However, I doubt I’ll be able to leverage this for the majority of work I do. Because of organizational restrictions, I won’t be able to upload and analyze our data in “the cloud”. An on-prem solution would be ideal. I understand the community is a huge goal of Observable, but I worry without an on-prem offering, you might be limiting your audience. Instead, I’m guessing many organizations will continue to use other notebooks like Jupyter.

Just my 2 cents. All this being said, keep up the fantastic work. I will try to use Observable as much as I can.

2 Likes

Folks, as you may have seen, we launched embeddable Observable notebooks this morning, and open-sourced the notebook-runtime.

On your published (or shared) notebooks, you’ll now find “ES module” and “Download tarball” links, which provide the notebook compiled down to an ES module, with all of its notebook dependencies bundled.

You can then run the notebook on your own website, or on Node.js, and observe specific variables — rendering them to the DOM, or doing something else computationally interesting with them. For the full details, see the introduction here:

This is a first stab at a minimal API for these, and I’m hoping to hear more from the folks in this thread about how we can improve it to make them more useful.

NB: @bumbeishvili — you’re quite right about wanting to hide some variables. By default, loading a notebook into the Runtime does nothing, and no cells are evaluated. You opt-in by attaching observers to the cells you’re interested in, and only those cells (and the cells that they depend on) will begin evaluation, reporting their values.

7 Likes

@jashkenas export, runtime, and inspector worked great in my first test drive!

Quick question about the export format: is looks like each variable in a module is represented as { inputs: [string], value: (...args) -> result }. Have you considered also including the (uncompiled) code from each variable’s cell? At the moment, it looks like variable.value.toString()–sans the wrapping function(){}–preserves the original code, but it seems like one wouldn’t want to rely on that (and that you wouldn’t want to guarantee it?).

Thanks!
-A

It’s not quite the original code from each variable’s cell. The Observable compiler needs to parse out your free variables, to assign as inputs (arguments) to the function body, and convert views and mutables into pairs of variables that are referenced by name. For example: viewof chart becomes a $0 argument and variable within your compiled cell.

We’ve considered also including the uncompiled source code in the downloadable notebooks — but that would be for the purpose of creating a save/load file format, which isn’t really the goal with these initial executable notebooks.

What exactly are you hoping to use the uncompiled code for? Backups? Or something else?

This is great—congrats! Here’s my feedback on the guide:

Versioning

Looks like the notebook’s version is now accessible via a header comment in the ES module:

https://api.observablehq.com/@shaunlebron/humanae.js?key=adb6ec0cc0b5322d

So we can use that to get a stable, versioned URL (e.g. 189):

https://api.observablehq.com/@shaunlebron/humanae@189.js?key=adb6ec0cc0b5322d

This seems useful, so maybe it should be mentioned in the guide?

Typos

  • Runtime.load(notebook, (cell) => <- needs an open-curly here
  • s/a just/just a/
1 Like

Thanks for the edits @shaunlebron — updated!

Cool, might be worth clarifying how to get a version number though—the only way I know is through React Devtools (search Af > state.publishVersion), but the ES Module now exposes it in the header comment:

// URL: https://beta.observablehq.com/@shaunlebron/humanae
// Title: Humanæ
// Author: Shaun Lebron (@shaunlebron)
// Version: 201  <----
// Runtime version: 1

Or maybe show it somewhere in the UI?

Yes, we definitely need a better way to show (and perhaps name) versions in the UI. That will certainly be a part of the improved notebook history / branching work we’re tackling, but we may do something smaller in the interim as well.

On this topic, it is absolutely not obvious in UI that the “share” button freezes a certain version number as “being public”, which then impacts what the next menu item does.

(On this topic) I couldn’t agree more, @Fil. That pretty much tops my list of personal peeves and priorities.

there’s no reason it wouldn’t work, but here it is…

1 Like

What exactly are you hoping to use the uncompiled code for? Backups?

@jashkenas I had two things in mind:

  1. Backups.
  2. Moving code into my own editor, for use as a non-Observable-based project (thinking of cases where code’s spread across enough notebooks/cells that copy-pasting from the UI is inconvenient). If the compiled function source is always intended to stay as close to the original code as possible, then the compiled code in the exported notebook would work fine for this… but if it were going to be compiled to a different language target (e.g. to ES5… or something less unlikely :slight_smile:) or minified, then having the uncompiled source would be useful.

Personally I would love to see an API where the whole notebook is fetched as a plain text file of uncompiled code, ideally with read/write access. Then I could edit the code in my own text editor and save changes which would sync back to the server. I could also use a wide variety of other tools such as diff tools, version control, grep, etc. etc.

Someone else might even want to make a simple FUSE filesystem for seeing all their notebooks in their file manager, etc.

5 Likes

@jrus looks like it is here
curl https://api.observablehq.com/document/@jrus/halton | jq .

4 Likes

Sorry for the newbie question, but I don’t quite grasp what I must do if, like it says there at the introduction of your most useful “Downloading and Embedding Notebooks”, I want to “save a notebook to my laptop, to show to my grandmother who lives in the internet-free woods of Nova Scotia”. I got how I can download a tarball of my code and also how to download the runtime and stylesheet that live observable’s servers, but unfortunately this seems to be not quite enough. For instance, in my offline index.html I replaced

import {Inspector, Runtime} from “https://unpkg.com/@observablehq/notebook-runtime@1?module”;

with

import {Inspector, Runtime} from “./runtime.js”;

where runtime.js is the file I downloaded from the url above. If I run my notebook from a server in my laptop, it runs ok if I have a network connection, but not in some internet-free woods. This particular notebook fails with

RuntimeError: md could not be resolved

and I guess any attempt to assess other observable goodies or other notebooks will similarly fail. How can I download everything I need and bring it with me to Nova Scotia?

Thanks!

@esperanc

You’re right!

I’m afraid that our current implementation of the md function, provided by the standard library, requires the marked library from Unpkg at runtime:

The notebook that I tried to run offline also depended on d3-format.

Although I could certainly imagine a version of the Observable runtime that bundled in marked, or a notebook that didn’t use it … neither of those feel like a satisfactory solution.

But I don’t think we’re going to be able to automatically determine all of the HTTP requests a notebook might make — or even just all of the names of libraries it might require — and scrape them in advance for your bundle … at least not any time too soon.

So, in lieu of a better solution in the short term, I’ve just gone ahead and reworded the introduction to remove any reference to “internet-free woods”…