Is the lack of visualization explaining fundamental notebook building brick - the stuff that is confusingly known as Inspector (beginners hate that one complex trick).
I know that all of you, who know what I am speaking about, are touched by the curse of knowledge. It looms over me too, and because the process of being consumed by it is very painful for my particular brain, I hope to still explain it to you here.
But then you want to do nice things. Convenient things. Things you’ve seen in other notebooks. And it is where the beautiful picture starts to fall apart.
Because nothing, almost nothing explains what is going on with the Notebook when you define cells, and then reference them. It is not just cells. It is two types of cells - “definition cell”, and “reference cell”. “definition cell” (where you assign value to cell variable) attaches stuff to Notebook, creating visualization, etc. “reference cell” (where you reference previous cell) doesn’t attach anything and invokes the Inspector that dumps info about variable, but not its viz.
But My Biggest Rant About Observable (that makes me feel insecure and stressed about trying to do things in Observable again) is two-fold.
I am still not sure I got it right (after like, being absent for some time)
I have to juggle all that stuff in my head, instead of meditating over some looped .gif animation until the enlightenment hits my cortex
P.S. Nevertheless thanks for a wonderful tool, and gratz for gathering Observable Insight. I hope this feedback will be valuable on the eve of the congression.
Press if you liked my insight, if you’ve felt the pain and if you’d give money to Observable to film the .gif story for enlightenment.
Think of each cell as a function that gets rerun automatically when any of its arguments change. The arguments to this function are the values of all other cells that the current cell references.
To see what these functions and their arguments (“inputs” in Observable speak) look like, check out the following link:
The same is true for non-Javascript cells. For instance, if you have a Markdown cell, it will be compiled to something like
function _using(md){return(
md`### A heading
Some text.`
)}
While I appreciate the explanation, it is still a lot of text, which I need to wrangle in my head.
Think of each cell as a function that gets rerun automatically when any of its arguments change.
That’s not always true. Fears in my head tell me there are cases when the function is not rerun if it was resolved once. But I don’t remember them.
I can’t give examples that I don’t understand, but here is the thing that I’ve failed to accomplish a year ago - get rid of red error messages if no input is given, here - List remote .zip contents / Anatoli Babenia | Observable
Another thing I wanted to do, is to control how many GitHub API to call, or call them manually once they queued. To avoid depleting my request quota, and show how it replenishes. To count active forks in a repo. Didn’t get far too GitHub API Starter (active forks) / Anatoli Babenia | Observable
Also an advanced usage pattern, and basically a vanilla JS throttling problem. If you want to go the extra mile, then the only addition here would be that a queued request might become invalid before it gets sent.
And despite all these explanations, the complete mental model of cells in Observable notebook just doesn’t compile in my head. That’s why I need visualization.
That’s a well known talk that inspired folks at Khan Academy and I won’t be surprised if people at Observable were driven by it too. Totally worth rewatching after 11 years to reflect how far we’ve got.
Ok. So revisiting this again after a while and trying to remember what it was all about. I can formulate this in one sentence as this.
Cell is a Lie
In Jupyter notebooks and in JavaScript console object introspection carries predictable behavior.
x in JavaScript console is always an int. I don’t expect surprises here.
I’ve got the same feeling of confidence with Jupyter Notebooks. I can’t remember when cell result would be dependent on the implicit variable type. Usually you have to explicitly ask to render something, like image (haven’t used Jupyter for a few years, so my memories may be bitrotten). No time to paste screens ATM, maybe later…
But all of that is not true with Observable cells. They are unpredictable. And therefore they need a diagram. Like cell rendering can vary if the variable inside is plain or viewof, if it awaits the input or not, if it was already used before or not. And things became more complicated if they are wrapped into {} blocks and functions, and imported, and awaited/asynced, and used with external JavaScript libraries.
I am not saying that things could be better. I am saying that things could be better documented, but I am not on a payroll to, so it is feel exhausting and unfair to do that.
No, haven’t seen that. Going to look at it now. Thanks. Although I admit I am not JS expert either. I can’t write the code to fetch page and process it like I would do in Python. Because fetch in JS is asynchronous, the code turns out to be complicated and hard to remember.
Yes. Visual reference without text, except for labels, would be ideal. I dk maybe a competition can help. Starting with a multi selection poll that asks if people understand cell model - its features.
Here’s an example that shows how to grab an HTML table with fetch, use D3 to select and process some tabular data, and create an output listing that tabular data:
Ultimately, my objective was to create a CSV file to share some information on named colors in Javascript with my data visualization students.
You’re absolutely correct that asynchronous programming is a bit trickier than synchronous programming. If you want to create dynamic images that render in a webpage after downloading data and respond smoothly to varying user input, though, then I think there’s really no avoiding asynchronous programming.
If anything, though, this type of programming is much simpler on Observable than not. In the notebook above, for example, I fetched the text of a webpage in one cell and then left it alone. Any subsequent cells I wrote depending on that text then wait to run until the download is complete, which is quite nice. I also used Observables html tagged template to translate the downloaded text into something that d3.select found palatable.
One of the major points that made me switch from classic notebooks to observable is reactivity. In jupyter notebooks for example, all is about the execution order (which is defined manually). So the state of a variable is not based on what you read but when it was executed, I often end up just running the whole notebook everytime to avoid errors. This is not a problem* in observable. Every value is effectively the value that you can see and that is propagated everywhere (the universe state of the notebook is coherent).
With the accumulation of concepts (promises, generators, rendering, databases, …). It augments the possibilities but notebooks becomes less readable. And I found myself debugging dataflows to just understand issues in my notebooks. On that point, I agree that non reactive systems are easier to use but I think that the inherent property of interactions forces us to add more complicated mechanisms.
As someone that has come back to old coding (because I don’t want to pay for private notebooks), there is a slowdown in the ease of development that observable allows.
I mostly disagree with this sentence, observable will* update to keep the universe coherent. a ipybn will not, you will end up with values that are false because you changed them before but forgot to re-execute this specific cell.
Finally, the difficulty to understand a notebook comes down to the person who created it. Some notebooks are very easy to understand and the logic can easily be extracted, some aren’t, but such is life. (for example, some of my notebooks are a nightmare to extract because the code is fully embedded in observable.
*: there is a lot of variability and these affirmation do not always hold true (looking at you input events)
Here’s an example that shows how to grab an HTML table with fetch, use D3 to select and process some tabular data, and create an output listing that tabular data:
In real world JS just fetching is not enough. Fetching errors need to be handled to give users actionable error messages to act on, whatever their WiFi is down, the URL is incorrect, the server returned error, or JSON de-serialization failed.
Clearly. This is exactly why I wrote “my objective was to create a CSV file”. I then used that CSV file as a FileAttachment in the demo that I built for my students.
I would go further and state that it’s probably a mistake to create any interactive visualization driven by data that’s dynamically scraped from another site. You literally wrote, though:
I can’t write the code to fetch page and process it like I would do in Python.
So, I thought I would share a basic example doing so. I guess you probably oughtta do that in Python, though.
Just in case it hasn’t been mentioned yet, we also have some great resources listed in the “Debugging” section of the Documentation, for example:
if you liked my insight, 
if you’ve felt the pain and 
if you’d give money to Observable to film the .gif story for enlightenment.
