Today, we’re super excited to announce Visual Dataflow: a new tool to help illustrate the reactivity that powers Observable notebooks!
We know that reading, writing, and editing notebooks can sometimes be confusing. You continuously update a carefully constructed mental model of the order in which your cells evaluate. You thoughtfully remember not only where each cell fits in that order, but also where it lives on the page. You painstakingly trace through the dependencies of cells as you write and refactor your code.
Now, we hope this all becomes a little (or a lot) easier with the help of Visual Dataflow and its two key features:
Our take on a minimap which, among other things, highlights the connections between the focused cell and other cells in the notebook
Special syntax highlighting for references to other cells in the notebook, paired with keyboard shortcuts to jump to the definition of these variables
These changes have been inspired by tons of feedback from tons of Observable authors. Above all, we want Visual Dataflow to be helpful to you, to enable you to continue writing amazing notebooks more quickly, intuitively, and delightfully than before.
But we know, as all good features are, this is work in progress and the only way we’re going to keep making it better is by listening to and learning from you. So check out our announcement notebook, play around with the new features, and let us know what you think — please, don’t hold back! We want to know what you love, what you don’t, what’s useful, and what’s missing.
seriously. this is just an awesome idea. can’t wait to use it.
edit: one thing i noticed. sometimes i want to do a “find and replace” e.g. to rename a variable. Maybe it is something than could be combined with visual dataflow without overloading it too much.
To preëmpt one particular potential thread of discussion: Currently, you need a fairly wide browser window to be able to use and enjoy the minimap.
We’re going to try to find a graceful way to allow the UI to coexist with the content, down to tablet and mobile-sized screens … but we don’t have it yet, so for now, go wide!
the 100px limit on cell names in the minimap cuts of some names (like percentInfectedGraphhere) depsite there being plenty of space on a wide browser window for the full name
being able to click where the name would go for unnamed cells is a really nice touch
It’s not really clear how to get out of the “seeing dependencies” mode. You can hit escape if the cursor is in the code section, but if you click on the dot in the minimap again, there’s no clear way to close the cell.
as I said in Slack, the red dots for errors will be really helpful for debugging
the background on the variables feels like a bit much for me, although it’s certainly possible I’ll change my mind after using it for a while. Maybe just a rounded border or an underline?
for reference, Atom uses cmd+click to go to definition, and Xcode uses cmd+click to open a little popup window. You can ctrl+cmd+click to jump to the definition, and option+click to show generated documentation.
One thing I noticed: if I open a long notebook in a collection and click on the name of a cell near the top of the minimap (e.g. the one labeled libmeta in this notebook), the dot moves to the top of the screen and then it gets obscured by the collection nav header.
While it’s on my mind again, I really don’t like how these floating headers (the collection one and the “Publish” one that shows up if I have unpublished / unshared edits) cover up notebook content when I click on anchor links in the notebook. It’s also kind of ugly when I link to a specific section of a notebook and half of the heading is covered up.
the first thought i had was to move a “local map” next to a cell. So that it is easy to see which dependencies a certain cell has. although i am not sure if this would solve the problem it feels like this could be a placing that would make sense.
I really like the “minimap” (table of contents) feature. I’m especially glad that it brings back cell names which were originally visible but have been hidden for a long time.
A few reflex comments:
I’m not convinced about scrolling it in proportion to the scroll of the full page, for very long documents with lots of cells. It’s annoying to have to scroll the whole page all the way up or down to skim through the full list.
It would be nice if clicking a dot for a named cell would put the appropriate anchor link into the URL
When I click on a dot, I imagine I generally won’t want to open the code for the cell if it is currently hidden. This behavior was a bit surprising, and makes it less useful as a table of contents for me. (Perhaps a simple click could just navigate to the cell and a modifier key + click could show the code.)
I’m not convinced about the blue background for symbols in code which are references to other cells. Is there a way to turn that off or tone it down for folks who find it distracting?
While I find this current blue background to be too strong, I do like the idea of identifying names which come from cells. It would eventually even be neat to e.g. also provide links from built-in names to MDN documentation or documentation for the observable standard library.
I wonder if de-emphasizing regular Markdown cells (i.e. md`...` with no interpolations or reactivity) would make the overview clearer. It’s really cool that Observable only needs one type of cell, but for example the Bar Chart Race, Explained notebook could likely be made a lot less bewildering on the minimap by hiding all the markdown cells.Maybe have a toggle in the cell dropdown menu to hide it from the minimap?
Off topic for this thread, but I also really dislike these (along with all similar floating headers/footers on websites in general, e.g. on this forum). (a) Vertical space is the main spatial constraint when reading in a browser on a laptop, and (b) browsers’ scrolling (etc.) behavior doesn’t interact nicely with floating banners covering part of the current viewport.
whoas wows ohhhhhhs (mutiple times—it’s really amazingly useful, and pretty!):
suggestions:
⌘cmd j to jump is great… but how to jump back? It’s easy to get distracted
is there a possibility to show a cell such as md'## title' as title instead of no name? (and +1 for demoting inactive md cells) [removed, I see that the idea is mentioned in Toph’s intro]
I like the fact the minimap gives you a general idea of the complexity of a notebook. Which leads to some remarks:
the minimap is very useful to understand the flow of dependencies in a small / toy notebook
it does not help so much on large notebooks, BUT it gives you a hint that, maybe, you should divide your notebook into smaller ones, and import them
so: I thought an improvement would be the detection and highlighting of autonomous subgraphs, that could be detached to a new notebook, and replaced by one import statement. A new feature could be to give the user a button to do this automatically → “Replace this subgraph by an imported notebook”
On the other side: it would be nice to be able to unfold the minimaps of the imported notebooks
I’m sorry, I won’t be able to give you any critical feed back on the new feature of a notebook overview that I just noticed today. It’s just too good. So I may simply encourage you to continue your amazing work which helps me enjoy life always a little more.
I wonder if de-emphasizing regular Markdown cells (i.e. md…`` with no interpolations or reactivity) would make the overview clearer. It’s really cool that Observable only needs one type of cell, but for example the Bar Chart Race, Explained notebook could likely be made a lot less bewildering on the minimap by hiding all the markdown cells.Maybe have a toggle in the cell dropdown menu to hide it from the minimap?
Or maybe more generically: toggle hiding all cells without any dependencies?
For those asking for a quieter cell reference treatment (@j-f1, @jrus), we’ve pushed some updates out this evening. References are now gentle underlines — option double-clicking and Command-J continue to work the same way. We’ve also greatly increased the performance of the minimap … hopefully to the point where it should be impossible to notice any overhead, no matter how many cells you have constantly flashing away.
To tack onto what Jeremy said, we’ve also allowed cell names to take up all available width to the right of the notebook, rather than capping it at 100px (@j-f1). And for those looking for a way to reset the minimap/stop seeing the dependencies of a cell, one quick tip is just to use the esc key — whenever a cell is focused, its dependencies will show, so unfocusing it by escaping should do the trick (@HarryStevens).
We’ve also opened tickets for (1) a shortcut key that will allow you to toggle back to the previously focused cell, (2) improving the experience in large notebooks by de-emphasizing markdown cells/cells with no dependencies and (3) refining the scroll behavior.
Thanks again, all, for the feedback and suggestions — if you think of anything else as you continue to use the minimap, keep it coming
Today, we’re super excited to announce Visual Dataflow: a new tool to help illustrate the reactivity that powers Observable notebooks! 
