# "Recompute with...", or: a way to "climb the ladder of abstraction"

**URL:** <https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008>\
**Category:** Help\
**Created:** [May 25, 2019, 6:00pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008 "2019-05-25T18:00:23Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 25, 2019, 6:00pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/1 "2019-05-25T18:00:23Z")

</div>

Hi,

First of all, congratulations for the site! The Observable notebook is an amazing tool for computing interactively. It truly feels like the best thing since spreadsheets, and I think that’s quite the achievement.

I’m here to ask: is there a way to take a certain cell of the notebook and “call” it (recompute its output) with a different value for certain other variables, maybe a few steps up in the DAG of dependencies?

This possibility would make the notebook even more attractive for building a deep understanding of algorithms and systems.

(An interesting document on the matter which got me thinking about Observable and motivated this post is Bret Victor’s [Up and Down the Ladder of Abstraction](http://worrydream.com/LadderOfAbstraction/), which is also the inspiration for the title.)

Thank you, and a million thanks again for this great tool!

---

<div class="post-metadata">

**Author:** ![bgchen](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bgchen/32/784_2.png) [@bgchen](https://talk.observablehq.com/u/bgchen)\
**Post date:** [May 25, 2019, 6:51pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/2 "2019-05-25T18:51:23Z")

</div>

There was some related discussion in [this thread](https://talk.observablehq.com/t/reusability-improvements/1512). One thought I’ve had since then is that you can do something along these lines via the notebook imports.

Suppose you have a notebook named `myNotebook` containing this cell:

```auto
cell = {
  // some code depending on cells input1, input2
}

```

Now, share `myNotebook` and take note of its version number `N`.

You can now write:

```auto
import {cell as cellCopy} with {newInput1 as input1, newInput2 as input2} from `myNotebook@N`

```

(Without pinning the version, the notebook will fail to work…)

See this little notebook for this idea in action:

> **[Cells as functions via self-imports](https://observablehq.com/d/f66cd3fbc95039ea)**
>
> An Observable notebook by Bryan Gin-ge Chen.

It should be possible to generalize this to more complicated dependency chains (though I agree with the posters in the above-linked thread that having this sort of thing more “built-in” would be cool to see…)

---

<div class="post-metadata">

**Author:** ![j-f1](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/j-f1/32/368_2.png) [@j-f1](https://talk.observablehq.com/u/j-f1)\
**Post date:** [May 26, 2019, 1:26am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/3 "2019-05-26T01:26:50Z")

</div>

Another option is to define the cell as a function, then call it with whatever values you want to change.

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 26, 2019, 7:20am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/4 "2019-05-26T07:20:09Z")

</div>

Right, that’s what I’m trying to do with my current notebook. But right now it’s tedious to refactor code into functions after going merrily with simple constant or cells, and the result is inflexible. Plus, in practice, one often feels the need to define which variables should vary after the fact anyway.

Could this be a neat new feature instead? (I’ll look up contributor’s guidelines in the meantime…)

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 26, 2019, 7:26am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/5 "2019-05-26T07:26:28Z")

</div>

Oh, cool! If I understand correctly, though, I’d need a new self-import for each new set of input values that I want to try. Which, if I understand correctly, means I can’t easily plot, for example, output values based on a range of inputs, right?

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 26, 2019, 7:28am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/6 "2019-05-26T07:28:27Z")

</div>

Just came to my mind: an interesting option could be for the runtime to expose enough of the dependency tracking and recomputation logic to allow the implementation of this feature from the notebook (like, by importing a lib/notebook). This, in turn, could enable a bunch of other interesting and unexpected features.

---

<div class="post-metadata">

**Author:** ![bumbeishvili](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bumbeishvili/32/4220_2.png) [@bumbeishvili](https://talk.observablehq.com/u/bumbeishvili)\
**Post date:** [May 26, 2019, 8:00am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/7 "2019-05-26T08:00:02Z")

</div>

Sadly that is not dynamic enough

- It’s not simultaneous if I change plus to multiplicity imported cell does not change right away

- If someone forks notebook, they still are referencing your notebook, not forked one

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 27, 2019, 8:25pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/8 "2019-05-27T20:25:23Z")

</div>

So… I started browsing the source code that is currently available (mostly [1], [2]), and I believe that there is no real way to extend the JS syntax and/or the notebook API in order to get this feature. I would have loved to try my hand at implementing this, but it seems that the relevant parts of the source code are still closed.

Am I mistaken?

[1] [https://github.com/observablehq/runtime](https://github.com/observablehq/runtime)  
[2] [https://github.com/observablehq/inspector](https://github.com/observablehq/inspector)

---

<div class="post-metadata">

**Author:** ![bgchen](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bgchen/32/784_2.png) [@bgchen](https://talk.observablehq.com/u/bgchen)\
**Post date:** [May 27, 2019, 8:39pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/9 "2019-05-27T20:39:31Z")

</div>

If you haven’t already, have a look at the Observable parser; if I understand what you’re after, you probably want to start by extending its functionality:

> **[observablehq/parser](https://github.com/observablehq/parser)**
>
> The Observable parser. Contribute to observablehq/parser development by creating an account on GitHub.

I made use of the parser in this notebook, which was my attempt to reproduce a small portion of the “closed-source” functionality:

> **[From raw notebook source to Observable runtime V1 modules](https://observablehq.com/@bryangingechen/from-raw-notebook-source-to-observable-runtime-v1-modules)**
>
> An Observable notebook by Bryan Gin-ge Chen.

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 27, 2019, 8:53pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/10 "2019-05-27T20:53:13Z")

</div>

Oh, damn! I don’t know how I missed it! Thank you very much!

Although, assuming I managed to extend both the runtime and the parser, I’d still have to write my own editor in order to test the change, right?

I guess I’ll keep reading the code to understand it better, in the meantime.

---

<div class="post-metadata">

**Author:** ![jashkenas](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/jashkenas/32/1778_2.png) [@jashkenas](https://talk.observablehq.com/u/jashkenas)\
**Post date:** [May 28, 2019, 4:39pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/11 "2019-05-28T16:39:59Z")

</div>

Great topic!

This is very much a concept that we’ve been wanting to take a crack at — and there has been a bit of discussion about some potential variations:

1. A local version of `import with ...` — as you all have described above: given a cell `chart` that depends on `height`, give me a different copy of `chart`, with `height` replaced by `mobileHeight`, or a literal value: `150`. You could use this at the top level, to display alternative variations of a cell … or within another cell like a function call, to produce local values, passing in different parameters.

2. Generalizing the above to iterables or arrays … wherein you’d be able to map a cell across an array of values for a given input, producing in this case an array of charts.

3. Wilder and more off-topic ideas, like nested “child” cells, or raw ways to reach in and rewire the dependency graph…

At this point, assuming that we want to take a crack at implementing #1, what would be most helpful to hear from you all would be a sketch of what you’d imagine your ideal syntax for this feature to be, in the context of Observable.

Should it look like an import? Like a function call? Like something else?

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 28, 2019, 5:14pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/12 "2019-05-28T17:14:26Z")

</div>

Hey there! Thank you for taking this into consideration.

My vote would go most enthusiastically to [this](https://talk.observablehq.com/t/reusability-improvements/1512/8) proposal made by @j-f1 in another, somewhat related thread.

It’s simple, it looks aesthetically and ergonomically “right” together with the rest of Observable’s syntax extensions to JavaScript, and most importantly, it does without the need for publishing and importing another notebook.

Although it seems important to me to be able to have one notebook contain just the algorithm “to be abstracted”, while another contains the “abstractions” (the experiments), I reckon it would be better if that wasn’t necessary. I think this way you would get most of the advantages of the “wilder” ideas from your point #3 with very few of the downsides. It would still be possible to understand the dependencies of a cell at a glance, and the “burden of the surprise” of playing the “trick” of overriding one of them would be on the cell that “wants” the override.

Btw, may I ask you if you can confirm whether or not the parts of the project that would need to be touched in order to implement this feature are not among the open sourced ones?

Thanks a lot again!

---

<div class="post-metadata">

**Author:** ![jashkenas](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/jashkenas/32/1778_2.png) [@jashkenas](https://talk.observablehq.com/u/jashkenas)\
**Post date:** [May 28, 2019, 5:31pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/13 "2019-05-28T17:31:28Z")

</div>

Thanks for the vote!

> [@bars](#):
>
> Btw, may I ask you if you can confirm whether or not the parts of the project that would need to be touched in order to implement this feature are not among the open sourced ones?

Most of the parts that would be needed to be touched in order to implement this feature are open sourced, but one is not.

To implement this feature, I think we’d need to update the `parser`, the `compiler` and the `runtime`. Of those, the `compiler` (although by far the smallest of the three) has not yet been open-sourced.

(It wouldn’t be strictly necessary, but we’d also want to update the editor, for syntax highlighting and fancy error messages)

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 28, 2019, 5:39pm UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/14 "2019-05-28T17:39:28Z")

</div>

Makes total sense. Thank you!

Although unfortunately JS is not _quite_ my forte, and I won’t have much free time until June 5, I’d still be delighted to contribute to the implementation effort, since this is a feature that I’ve wished for so strongly lately.

Keep up the (really) good work!

---

<div class="post-metadata">

**Author:** ![bumbeishvili](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bumbeishvili/32/4220_2.png) [@bumbeishvili](https://talk.observablehq.com/u/bumbeishvili)\
**Post date:** [May 29, 2019, 6:46am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/15 "2019-05-29T06:46:31Z")

</div>

My vote:

If we have the following cells

```javascript
a = 2

```

```javascript
b = 3

```

and we also have

![38%20AM](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/a/a317e37412e47505b31228921e08b1c29a3bf2f6.png)

My desired syntax for `y`'s generalization would be following (Ordered by priority)

#### Syntax 1

```javascript
yCallable = Notebook.callable(y) // will return function

```

then we can write  
 ![19%20AM](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/a/abda37e942139636378abb8e5b086d8c27a1ba92.png)

Right now it does not seem to be possible, but it also would be good if we could inspect function arguments (To see that `a` has a default value of 2 and `b` has a default value of 3)

![25%20AM](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/4/47b34559ad67f517ac1ead1307dac1ad90c79690.png)

#### Syntax 2

Exactly like @mbostock mentioned

```javascript
functionof y( {a: 42} ) // b = 3 is implicit

```

#### Syntax 3

Just

```javascript
call y({a: 42}) // 

```

#### Syntax 4

Syntax by @j-f1 , it’s good also

```javascript
y with { a: 42 } // b = 3 is implicit 

```

#### Syntax 5

Another suggestion by @j-f1 , which I don’t like because `y` is a string and we lose dependency tracking

```javascript
Notebook.call('y', { a: 42 })

```

Maybe just

```javascript
Notebook.call(y, { a: 42 }) // without string

```

Actually, this could be merged with syntax1

# My arguments against new syntax

- In future we may have other meta stuff related needs and requests, I don’t think creating new syntax for each one would be good

---

<div class="post-metadata">

**Author:** ![j-f1](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/j-f1/32/368_2.png) [@j-f1](https://talk.observablehq.com/u/j-f1)\
**Post date:** [May 29, 2019, 10:07am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/16 "2019-05-29T10:07:47Z")

</div>

Very interesting! IMO the versions that entail a new syntax are better since using an API call either involves passing the name of the variable as a string (which doesn’t create a dependency, as you pointed out) or involves passing the variable as an argument. That’s confusing because in JS you can’t get the name of a variable that’s been passed as an argument so it’d require a syntax transform that would make one function work in a way that no normal function does, which sounds like a keyword to me.

---

<div class="post-metadata">

**Author:** ![bumbeishvili](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bumbeishvili/32/4220_2.png) [@bumbeishvili](https://talk.observablehq.com/u/bumbeishvili)\
**Post date:** [May 29, 2019, 10:38am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/17 "2019-05-29T10:38:27Z")

</div>

I believe Observable runtime preprocess these variables before it’s displayed, so when we pass cell variable as an argument (to notebook function) it will know what kind of object got passed (and certainly will know a name)

This is cell’s object representation

```javascript
 {
      name: "y",
      inputs: ["a","b"],
      value: (function(a,b){return(a+b)})
 }

```

---

<div class="post-metadata">

**Author:** ![j-f1](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/j-f1/32/368_2.png) [@j-f1](https://talk.observablehq.com/u/j-f1)\
**Post date:** [May 29, 2019, 11:16am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/18 "2019-05-29T11:16:25Z")

</div>

That’s true. However, when the cell is being run, all that’s provided to the cell is the current value of the variable.

---

<div class="post-metadata">

**Author:** ![yurivish](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/yurivish/32/198_2.png) [@yurivish](https://talk.observablehq.com/u/yurivish)\
**Post date:** [May 30, 2019, 12:53am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/19 "2019-05-30T00:53:56Z")

</div>

It would be great to have a syntax that can be composed with other code, rather than only being allowed as a top-level expression.

And at first I thought this looked weird, but now I like it:

```javascript
f(y with { a as 42 })

```

It’s short, makes it clear that what’s going on is a little bit unusual (not quite metaprogramming but in the same spirit), the `with` form evaluates to a value so it plays nice with the rest of the language, and the use of keywords enforces that “`y`” and “`a`” refer directly to cell names.

One consequence of this syntax is that it prevents abstracting the name of the cell, as the string `y` is semantically significant. Maybe the `with` form can desugar to a call to `Notebook.callable` that takes the cell name as a string and the overrides as an object, so the power is there for those who seek it.

Edit: If the `with` form evaluates to a value, it can be used with `map`: `all_the_data.map(d => chart with { data as d })`

---

<div class="post-metadata">

**Author:** ![bars](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/bars/32/1768_2.png) [@bars](https://talk.observablehq.com/u/bars)\
**Post date:** [May 30, 2019, 9:19am UTC](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008/20 "2019-05-30T09:19:17Z")

</div>

> [@yurivish](#):
>
> If the `with` form evaluates to a value, it can be used with `map` : `all_the_data.map(d => chart with { data as d })`

For me, this is what it’s all about! From my perspective, the point is to take a part of the computation and explore how its output/behavior changes in reaction to many inputs, maintaining composability and expression semantics, with no “context change”.

Limiting it to cells names (and not variables more generally) looks to me even more correct: it would force dependencies/inputs (and their computation) to be explicit, evident, a 1st class component of the notebook (I guess this is where _observable_ comes from?)

[Next page](https://talk.observablehq.com/t/recompute-with-or-a-way-to-climb-the-ladder-of-abstraction/2008.md?page=2)