First of all, congratulations for the site! The Observable notebook is an amazing tool for computing interactively. It truly feels like the best thing since spreadsheets, and I think that’s quite the achievement.
I’m here to ask: is there a way to take a certain cell of the notebook and “call” it (recompute its output) with a different value for certain other variables, maybe a few steps up in the DAG of dependencies?
This possibility would make the notebook even more attractive for building a deep understanding of algorithms and systems.
(An interesting document on the matter which got me thinking about Observable and motivated this post is Bret Victor’s Up and Down the Ladder of Abstraction, which is also the inspiration for the title.)
Thank you, and a million thanks again for this great tool!
There was some related discussion in this thread. One thought I’ve had since then is that you can do something along these lines via the notebook imports.
Suppose you have a notebook named myNotebook containing this cell:
cell = {
// some code depending on cells input1, input2
}
Now, share myNotebook and take note of its version number N.
You can now write:
import {cell as cellCopy} with {newInput1 as input1, newInput2 as input2} from `myNotebook@N`
(Without pinning the version, the notebook will fail to work…)
See this little notebook for this idea in action:
It should be possible to generalize this to more complicated dependency chains (though I agree with the posters in the above-linked thread that having this sort of thing more “built-in” would be cool to see…)
Right, that’s what I’m trying to do with my current notebook. But right now it’s tedious to refactor code into functions after going merrily with simple constant or cells, and the result is inflexible. Plus, in practice, one often feels the need to define which variables should vary after the fact anyway.
Could this be a neat new feature instead? (I’ll look up contributor’s guidelines in the meantime…)
Oh, cool! If I understand correctly, though, I’d need a new self-import for each new set of input values that I want to try. Which, if I understand correctly, means I can’t easily plot, for example, output values based on a range of inputs, right?
Just came to my mind: an interesting option could be for the runtime to expose enough of the dependency tracking and recomputation logic to allow the implementation of this feature from the notebook (like, by importing a lib/notebook). This, in turn, could enable a bunch of other interesting and unexpected features.
So… I started browsing the source code that is currently available (mostly [1], [2]), and I believe that there is no real way to extend the JS syntax and/or the notebook API in order to get this feature. I would have loved to try my hand at implementing this, but it seems that the relevant parts of the source code are still closed.
If you haven’t already, have a look at the Observable parser; if I understand what you’re after, you probably want to start by extending its functionality:
I made use of the parser in this notebook, which was my attempt to reproduce a small portion of the “closed-source” functionality:
This is very much a concept that we’ve been wanting to take a crack at — and there has been a bit of discussion about some potential variations:
A local version of import with ... — as you all have described above: given a cell chart that depends on height, give me a different copy of chart, with height replaced by mobileHeight, or a literal value: 150. You could use this at the top level, to display alternative variations of a cell … or within another cell like a function call, to produce local values, passing in different parameters.
Generalizing the above to iterables or arrays … wherein you’d be able to map a cell across an array of values for a given input, producing in this case an array of charts.
Wilder and more off-topic ideas, like nested “child” cells, or raw ways to reach in and rewire the dependency graph…
At this point, assuming that we want to take a crack at implementing #1, what would be most helpful to hear from you all would be a sketch of what you’d imagine your ideal syntax for this feature to be, in the context of Observable.
Should it look like an import? Like a function call? Like something else?
Hey there! Thank you for taking this into consideration.
My vote would go most enthusiastically to this proposal made by @j-f1 in another, somewhat related thread.
It’s simple, it looks aesthetically and ergonomically “right” together with the rest of Observable’s syntax extensions to JavaScript, and most importantly, it does without the need for publishing and importing another notebook.
Although it seems important to me to be able to have one notebook contain just the algorithm “to be abstracted”, while another contains the “abstractions” (the experiments), I reckon it would be better if that wasn’t necessary. I think this way you would get most of the advantages of the “wilder” ideas from your point #3 with very few of the downsides. It would still be possible to understand the dependencies of a cell at a glance, and the “burden of the surprise” of playing the “trick” of overriding one of them would be on the cell that “wants” the override.
Btw, may I ask you if you can confirm whether or not the parts of the project that would need to be touched in order to implement this feature are not among the open sourced ones?
Most of the parts that would be needed to be touched in order to implement this feature are open sourced, but one is not.
To implement this feature, I think we’d need to update the parser, the compiler and the runtime. Of those, the compiler (although by far the smallest of the three) has not yet been open-sourced.
(It wouldn’t be strictly necessary, but we’d also want to update the editor, for syntax highlighting and fancy error messages)
Although unfortunately JS is not quite my forte, and I won’t have much free time until June 5, I’d still be delighted to contribute to the implementation effort, since this is a feature that I’ve wished for so strongly lately.
My desired syntax for y's generalization would be following (Ordered by priority)
Syntax 1
yCallable = Notebook.callable(y) // will return function
then we can write
Right now it does not seem to be possible, but it also would be good if we could inspect function arguments (To see that a has a default value of 2 and b has a default value of 3)
Very interesting! IMO the versions that entail a new syntax are better since using an API call either involves passing the name of the variable as a string (which doesn’t create a dependency, as you pointed out) or involves passing the variable as an argument. That’s confusing because in JS you can’t get the name of a variable that’s been passed as an argument so it’d require a syntax transform that would make one function work in a way that no normal function does, which sounds like a keyword to me.
I believe Observable runtime preprocess these variables before it’s displayed, so when we pass cell variable as an argument (to notebook function) it will know what kind of object got passed (and certainly will know a name)
It would be great to have a syntax that can be composed with other code, rather than only being allowed as a top-level expression.
And at first I thought this looked weird, but now I like it:
f(y with { a as 42 })
It’s short, makes it clear that what’s going on is a little bit unusual (not quite metaprogramming but in the same spirit), the with form evaluates to a value so it plays nice with the rest of the language, and the use of keywords enforces that “y” and “a” refer directly to cell names.
One consequence of this syntax is that it prevents abstracting the name of the cell, as the string y is semantically significant. Maybe the with form can desugar to a call to Notebook.callable that takes the cell name as a string and the overrides as an object, so the power is there for those who seek it.
Edit: If the with form evaluates to a value, it can be used with map: all_the_data.map(d => chart with { data as d })
For me, this is what it’s all about! From my perspective, the point is to take a part of the computation and explore how its output/behavior changes in reaction to many inputs, maintaining composability and expression semantics, with no “context change”.
Limiting it to cells names (and not variables more generally) looks to me even more correct: it would force dependencies/inputs (and their computation) to be explicit, evident, a 1st class component of the notebook (I guess this is where observable comes from?)
