Metaprogramming with Peek/PeekFirst

I have had a few places where I wanted to programmatically access another notebook’s cells. I think it’s quite a good way to add generic features to the ecosystem (e.g. SVG to GIF) without forcing people to change their notebooks to accommodate them.

Anyway, since I have implemented this poorly twice, I tried to implement a good standalone one for reuse.

This was better than previous attempts because in this new version, peek returns a generator (i.e. stream of values), which is, perhaps not immediately obviously though, the right representation.

Previously in all my applications I so far I just wanted to read a single value, so made bad hacks to do that. For reading a single value I have peekFirst resolve to a promise, but in the general case it should be implemented with generators. Its trivial to turn a generator to a promise, but impossible the other way around, and the general case kinda mandates generators.

Anyway, I personally don’t find generators easy to work with, especially the nuance like passing an error through a generator, so I hope people will find value in this notebook. It’s done the right way, even error handling. You probably just want to use peekFirst in most cases though.

Let me know if you want more features. I could, e.g., add the ability to inject values into the target notebook before peeking.

4 Likes

very cool! that reminds me of this

Oh geez its like exactly the same thing with the one missing feature I was ruminating over. I been looking all over for something like this. I actually started planning from Notebook Visualizer / Observable / Observable

I even had a really crappy version that scraped the api endpoint textually. importCell! ARGH! I guess I only wasted 1 day of my life. We definitely need better discovery.

3 Likes

*Spent 1 day learning. :wink:

3 Likes

Exactly! in addition to mastering how Observable runtime works

2 Likes

Thanks!!! I updated Create an histogram in one line of code / Sylvain Lesage / Observable.

1 Like

(notice while refactoring) It’s subtle but Mike’s is an async generator and mine was a plain generator. I think I prefer a plain generator because await g.next().value vs. await (await g.next()).value. We both use Generator.observe which returns a stream of promises. The additional outer layer of async is not strictly needed.

If the generator is piped into a cell expression anyway, there are no real differences.

EDIT: PR Comparing ‘Dataflow’ by Mike Bostock to ‘Dataflow’ by Tom Larkworthy

@tomlarkworthy What about

// In many cases the imported cell will only have a single value, but
// we must use the most generic representation (an async generator) as
// the imported cell may be an async generator, or may reference one.

/pedant

Actually both implementations are suboptimal because they both use generator.observe which is a generator of promises.

It’s worth thinking about the difference between async generator of concrete values vs generator of promises.

In the async generator case the iterators ‘done’ and 'next 'controls are behind a promise. This accurately reflect you don’t know what will happen in the future and presents the caller from calling next until time has passed. But when you do know, you know the value at the same time as knowing the iterators next step in the control flow.

With a generator of promises only the value is behind a promise. The caller can iterate multiple steps and resolve the values later. So this is an abstraction that works for errorless infinite sequences.

So the ideal signature is a async generator of concrete values (or throw errors). Which neither provides and they can’t if they use Generator.observe. we need AsyncGenerator.observe (see Generators.asyncInput? · Issue #126 · observablehq/stdlib · GitHub)

1 Like