feature idea: import cached cells

I was looking at https://observablehq.com/@tmcw/why-observable-for-people-who-already-have-a-text-editor and https://bost.ocks.org/mike/make/ recently, and thinking about some of my own notebooks, and I realized that one still-sometimes-frustrating thing about Observable is that some notebooks have very high precomputation-at-startup latency (and if authors aren’t delegating to web workers, etc., the main UI thread can be stalled causing the browser UI to get glitchy and unresponsive). This can be a real killer on low-end devices.

If running (e.g. jupyter) notebooks or other programs locally, it is often convenient (with relatively low friction) to cache partially computed results while working on later steps. But in Observable this gets inconvenient.

I realize there are tricky considerations around hosting costs, the dynamism of DOM elements, etc., but it would be a really neat feature if we could import cached {foo} from "@user/bar" and have the Observable servers run the notebook once to create the content of foo, and then subsequently just serve up the computed data (html fragment, image, big binary data file, …) instead of needing to manually export the cell output, rehost it at some other site, and then load it via the fetch API or whatever, remembering to repeat the process any time the cell’s computation changed.

Or maybe even a way to specify the same “get this from cache if possible” logic within the syntax of a cell, perhaps with a cached keyword similar to viewof.

Anyway, this is not a short-term demand, but just an idea for you folks to think about as an addition in the months or years ahead.

4 Likes

For that matter, caching doesn’t have to be on the server side to be useful. Being able to cache data locally in the browser in a streamlined way would also be helpful.

I haven’t thought too much about it; maybe someone wants to take a crack at wrapping up https://developer.mozilla.org/en-US/docs/Web/API/Cache or https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API/Using_the_Web_Storage_API in a way that makes it fast and easy to wrap a cell.

A related feature under development is the ability to attach files to notebooks. (The files will be part of the notebook history, private for private notebooks, and copied on fork.) So if you have an expensive computation, you could download the output of a cell as a file (e.g., JSON or CSV) using the cell menu, and then drag it back onto the notebook to upload it and save its contents. That’s not quite as convenient as automatic caching, but it has many other applications and we can consider adding further conveniences in the future. Hopefully it will be available soon!

8 Likes