Algoritma of Observablehq.

Hi there.,

Observablehq is what i am looking for as webpage builder on the fly.
However, i am curious on how does observablehq works.

I know observablehq use codemirror as the main input but how to connect it with observablehq code (runtime, inspector and stdlib, etc)?
What does cell mean in the notebook?

Thanks in advance.

One of the key pieces of the observable editor is a (closed-source) compiler which translates the code in the cells of an Observable notebook into ā€œvariablesā€ of a ā€œmoduleā€ (to use the terminology of the runtime). The output of the compiler is the code that you can view by going to https://api.observablehq.com/[your-username]/[your-notebook-name].js?v=3

For example, compare this notebook:

with the compiled output at https://api.observablehq.com/@observablehq/five-minute-introduction.js?v=3

By the way, when you import a notebook, the Observable runtime is actually importing this compiled output and creating a new ā€œmoduleā€ inside your notebookā€™s runtime.

@asg017 made a repo with an open-source version of the Observable compiler here. You can play around with it here:

3 Likes

Thanks, thatā€™s the beginning for the next stepā€¦
I will use and follow that codeā€¦

Hi bgchen,

I have successfully reproduce the observablehq compiler from editor (codemirror) to render the Code into the Result Cell (refering to html created to show the result)ā€¦
Now i want to create a glue that connect or link The Code on the editor with the Result Cell.

The option are:

  1. Every Result Cell will always have an editor that keeping The Code exist or.

  2. The Editor will be showed on edit session only while The Code are parsing back from Result Cell.

Or are The Code already saved on somewhere that i donā€™t know yet. And if yes how to parsing back or retrieve that code back to the Editor.

Please give me your clue.
Thanks in advance

Sorry, Iā€™m afraid I donā€™t understand your question. Do you have some code or a notebook you can share?

My code still in local server, how to share the live with you?

But after i read once more time on ā€œunofficial-observablehq-compilerā€ there is one method named compile.notebook(object) which can generate notebook like what i am looking for the same as observablehq notebook.
object are array of cell objects with the form like this
{ nodes: [ { value: "a = 1" }, { value: "b = 2" }, { value: "c = a + b" }] }

my questions are just how to keep value of each cell linked or bonded into each editor or each html element. I think it is simple like just get the value from editor and stored in javascript object or localstorage. Where before i read compile.notebook method i cannot figure it out using compile.module.

However, if you explain me it would be better.

Thanks in advance.

Not sure if youā€™ve seen this, but there is an official open source version of the runtime: https://github.com/observablehq/runtime

@shancarter - @bgchen linked the official runtime in his response; I think the topic concerns the compiler?

gotcha, apologies.

One thing I do want to make clear is all the different tools that are in play when re-making your own notebook environment:

  • The Runtime: The internal engine that powers cell reactivity + modules. It takes cell names, dependencies, and cell definitions (as JS functions) as inputs, and outputs a reactive program of cells (the ā€œnotebookā€ weā€™re used to).
  • The Inspector: A ā€œrendererā€ that renders the output of the Runtime onto the DOM of a webpage.
  • The Parser: a parser that parses ā€œObservable JavaScriptā€ into Acorn syntax trees.
  • The Compiler: A tool that compiles ā€œObservable JavaScriptā€ into inputs that the Runtime expects, using the parser internally. Like @bgchen said, there is an unofficial compiler that I started a while back, but no open-source ā€œofficialā€ version.
  • The Notebook: The frontend notebook that glues all of the above together, for a live-coding experience. there is no open-source version of this out there (that I know of).

@Srabutdotcom, youā€™re making ā€œThe Notebookā€ part of the above. Itā€™s not an easy task, but totally doable!

re your question about which compiler method to use, it seems like youā€™ll want to have a ā€œvariableā€ (in runtime lexicon) for each ā€œcellā€ (in notebook lexicon). Each cell would have a CodeMirror text editor, a <div> where the runtime renders the output of the cell (using the Inspector). For this, compile.cell could be helpful - itā€™ll return a define and redefine function that you could you to instantiate and redefine the cell/variable.

Another option, if you want to instead have 1 large CodeMirror text editor for the entire notebook, is to use compiler.module, which is what I did for atom-observable, which is a Atom extension to render an Observable notebook locally.

As far as how to coordinate codemirror text editors and cell outputs, thatā€™s a bit trickier, itā€™ll depend how youā€™re building the app. If itā€™s a React app I bet you could componentize it all to make it a bit easier (which is what observablehq.com does, I think), but feel free to share a github repo or gist of your code!

5 Likes

thanks @asg017 for the explanation,

i have made a repo for you to review at observablehq-like-notebook
my first concern are, the Inspector always create new element instead of revising the Result Cell (id="render).

Any suggestion ?

I am able to reconstruct define and redefine as @asg017 advised. thanks.
I will think for the next step, to create editor in so we can edit each cell on observableā€“inspect cell created by inspector.

You can review my repository at obscompiler.
My previous repository was deleted please ingore it.

Hi @asg017 and @bgchenā€¦

I have other problem, where when iā€™ve compiled a cell let say variable ā€œaā€ and of course stored to modul. The case are when i renamed the variable ā€œaā€ to let say ā€œbā€ by compiling cell to modul, the previous ā€œaā€ still exist on that modul. And i have no idea on how to delete variable ā€œaā€.

in Standard observablehq variable function this can be done by storing the defined variable into any constant variable and we can delete by deleting that constant i.e variable.delete(). But how do we delete that partial variable already stored into modul?

If I understand you correctly, I would just use the variable.define method to redefine the variable from a to b (with the corresponding definition).

@bgchen. no. the point are how to delete existing variable that aready defined using let say compiler.cell("a=10") then i want to delete that "a" from runtime.module.

It is different case with define or redefine where we change the value of variable "a" to "b" where i do understand and no problem with this.

1 Like