Officially announcing the new Observable

[tophtucker]

The new Observable is now the default, and Mike has published his official announcement:

For more than a year, we’ve been quietly building an all-new Observable. Today, that new Observable is live on observablehq.com.

“All-new” is not an exaggeration. We ignored the sensible maxim never rewrite from scratch; only our database and open-source runtime survive the rewrite!

Why undertake such a foolhardy effort? In short, we want Observable to be great. We aspire to deliver the best experience and anything less leaves us intensely dissatisfied. We don’t want more features — we want fewer distractions. We want focus and clarity. We want Observable to be a beautiful, finely-crafted, purpose-driven tool that turns your imagination into reality.

But also: AI. AI has enabled a vastly wider audience to wield code. Rather than obviating the need to understand code, we believe our mission to make code more accessible is more urgent than ever. We reimagined how computational notebooks could work with agents as the primary interface. Our new agent-first notebooks, called chats, are radically different from our human-first notebooks and yet seamlessly interoperate with them. You can pick the mode of development you like best, and move between modes fluidly.

(I’d especially draw your attention to the last section, “I’ll end with a few personal words”…)

The complete old interface remains accessible at https://old.observablehq.com/. It remains the only way to edit collections, manage your team, view version history, revert, transfer, set granular access control, configure databases and secrets, schedule notebooks, see the minimap, etc. It’s a painfully long list; we know we’re cutting over pretty early. As ever, your feedback is invaluable to help us prioritize.

In June we “soft-launched” the new Observable here in the forum, and with your help we’ve shipped lots of changes. But so far it has represented only about 4% of usage. Almost all Observable users didn’t even know “the new Observable” existed. To focus our efforts, we have to bring many more of them along.

As ever, we’re here to listen! Thank you so much for paying attention and speaking up in the forum, which pays dividends for the vast silent majority of users.

[jwb]

Hrmm. The new files interface seems to lack some of the features of the old. It was obvious before which file to replace. Now it’s just a huge undifferentiated list of all the file versions that were ever in the notebook. How am I meant to deal with this? As seen at https://observablehq.com/@jwb/berkeley-rent-board-data

[mbostock]

What you’re seeing is really an artifact of the old Observable design which did not allow you to replace files; you could only add new files with new names (hence the @2, @3, @4, etc.).

In new Observable, when you replace a file, it actually gets replaced (for subsequent versions of the notebook)! You can also delete files without breaking old versions of the notebook. And you can rename files to whatever you like (again, without breaking old versions, and it automatically updates your code to match).

Old Observable tracked which files you referenced to show “unused” files because of the limitation that you couldn’t rename or replace files, or delete them without breaking old versions. We could show you which files are unused in new Observable, too, to help with the transition, but ultimately we don’t believe it’s necessary because there’s no reason to accumulate unused files in Observable (as you can actually replace and delete files).

You can also access the old interface at old.observablehq.com, if it helps.

[lenaMK]

Hi, I’m searching for a couple features as well.

I have looked at the documentation collection but didn’t see any of these answers there. A major update 1 week before I start teaching with Observable is a little shock, sorry if I’m just blind-sighted by old habits. Also, I hope to be able to do everything without AI, I understand it’s integration but I hope to retain the option to work and especially teach without it.

• For newbies, the options to start a cell with Tables or Plots was great to reduce “fear of the blank page and of javascript”, are these features still somewhere (without using the AI chat → btw, when I accidentally clicked on the chat, I found no way to exit the chat… no close button, escape or ctrl+c doesn’t work)

• Are the minimap and find pane still accessible somewhere?

• How can I link to a cell? Are any of the options that used to be next to the cell still available?

  Where can I access the version history of my notebook?

• Can I still set an order for a notebook collection ?

• Is there still a way to browse & search all notebooks in Observable? It was a great way for students to discover various styles of coding and visualizations that are close to their topics of interest

Please let me know if there is a more appropriate space for this type of feedback, and thank you!

[mbostock]

Thanks for the feedback, @lenaMK. And sorry for the bad timing!

AI is entirely optional; you’ll be able to teach without it. We think AI is a helpful tool for learning (when used judiciously) but completely understand any reservations you may have.

We’ve no plans to implement non-code cell modes (data table cells and chart cells) in new Observable. While these cell modes were convenient and arguably friendlier for some users, they were extremely limited and added a lot of complexity to the user interface and implementation. We found these cell modes to be functional “dead ends” for users rather than helpful stepping stones to learning. And these modes saw relatively little use overall. Observable is a computational notebook and by nature a code-centric tool. Especially with AI (optionally), we think it’s better to embrace code. And we think this focus on code and parsimony in features makes Observable better — though again, sorry for the disruption.

That said, SQL cells retain a lot of the functionality of data table cells (just with SELECT * FROM table). We haven’t yet shipped an improved summary table for SQL cells, but we’re working on it and it should be available soon. This will provide visual column summaries and visual filtering for tabular data without writing code (other than SQL). We think the summary tables are super useful so it’s worth the effort; meaning, the point of the summary tables isn’t to eliminate the need for code, but to give you something better than what you can practically write with code.

The way to exit a chat is the same as exiting a notebook: you choose to go somewhere else. You can go back to your workspace using the workspace menu, create a new notebook, etc. Perhaps we should add a “back to workspace” button next to the workspace menu. It’s probably worth the space given how common this action is even though it’s redundant with the menu.

We haven’t yet implemented linking to cells. Notebooks 2.0 have slightly different structure since we’ve adopted vanilla JavaScript and TypeScript, so we’re planning on supporting both links to Markdown headings (the anchors are already present using markdown-it-anchor via Notebook Kit, but links don’t yet work due to the iframe sandbox) and to named top-level variables (which is roughly equivalent to cell names in Notebooks 1.0).

The dataflow panel (a.k.a. minimap), find panels, and history panels aren’t yet available in new Observable, but we’re working on it and they will be coming soon. You can access version history in old Observable for now. Similarly new Observable doesn’t yet support editing collections, nor search, so you’ll need to use Old Observable for that for now. Though of course you can browse public notebooks and collections and authors to get inspired.

We’re working on building out more features as fast as we can, and we appreciate you letting us know which features are most important to you as that helps us prioritize (as well as feel some extrinsic validation)! Thank you!

[tophtucker]

@lenaMK I’d love to get on a call anytime if it’d help unblock you & your students!! I know this is bad timing with the start of the semester and I’ve been meaning to reach out to more educators (if I can find them all). DM here or email toph@observablehq.com what time & email would be good for you.

[ppierre]

Hello,

https://observablehq.com/@observablehq/sample-datasets have his anchor link broken “#-aapl” → “#aapl”

I used this page for making my students to download a JSON version of the data. The new “export” of the “cell toolbar” permit only PNG !

Can I expect a future JSON version along those existing (SVG and PNG ?) ?

Exporting to JSON is important it help use Observable to prepare data for other use.

[hari]

We can see in the anouncement a screenshot of the new interface where some new supported languages seems to be displayed (R, Python, etc..).

Is this an hint/easter-egg of Observable upcoming support to more languages ? That would be awesome

Announcement picture

My editor picture

[mbostock]

Yes. Those languages (currently called data loaders) are currently supported by Notebook Kit and Desktop. As they do not run natively on the web, we need to build a bit more infrastructure to support hosted compute before we can offer them on observablehq.com. I can’t give you an exact timeline but hopefully later this year.

[rusosnith]

yes please! the old file manager section was quite good already, the new direction got rid of lots of usefull features.

[neocarto]

@mbostock

Yes. Those languages (currently called data loaders) are currently supported by Notebook Kit and Desktop. As they do not run natively on the web, we need to build a bit more infrastructure to support hosted compute before we can offer them on observablehq.com. I can’t give you an exact timeline but hopefully later this year.

Just to make sure I understand how these data loaders will work: will they run on the developer’s machine and be executed at build time? But then, what happens if someone forks the notebook? And what if I move the data to a different location on my computer, etc.?

I’m having a hard time imagining how all of this will work on the platform. But I imagine you’ve already thought through all these issues.

In any case, I’ve been waiting for this for years!

[mbostock]

Yes, as I noted above we haven’t implemented linking to cells (or variables or headings) yet, but we’re working on it.

We do also plan on extending the export menu to include data exports (JSON and CSV, maybe Arrow and arbitrary Blobs too). That should be fairly quick to do.

Thank you for the feedback!

[mbostock]

Hypothetically, when using Observable on the web, data loader cells will run in the Observable cloud and not on your local machine. It’s similar to how scheduled notebooks work in notebooks 1.0 (which we also plan to support for notebooks 2.0), although it’ll be more real-time since you’ll obviously want to be able to edit these cells, evaluate the code, see what happens, and iterate. (You’ll also be able to run these cells locally using Notebook Kit or Desktop; we don’t plan on changing that.)

[neocarto]

Okay, that sounds incredible. I can’t wait to try it!

Another thing.

Will you keep the contextual menu that allowed users to add charts, inputs, and many other things (which was very useful for students)? Or will we now have to ask the AI to do these things instead (which, in my opinion, is not a good idea)?

Will you bring back the “Embed” menu as it was before?

In any case, thank you again for this incredible work.

And thank you for taking the time to answer questions on this forum. It’s really much appreciated.

[rusosnith]

What should we do with “previously working notebooks that don’t work anymore” situations? I don’t even see any error in the new notebook itself. Gladyl, the old one still works.

Should we submit them anywhere for you to have a look at?

[jwb]

@mbostock

We’ve no plans to implement non-code cell modes (data table cells and chart cells)

I’ll be mourning that, too. Data table cells provided a quick and easy way to attach a new file, flip between different files, judge data quality at a glance, and do common column transformations without a code cell.

[ppierre]

Many thanks for future JSON export, very much needed.

Link (anchor) to cell work it just the convention that have changed :

[aapl](#-aapl)

Should be now :

[aapl](#aapl)

Without the “-” (initially for the space at the title ?

# aapl

it is in : https://observablehq.com/@observablehq/sample-datasets

[mbostock]

@neocarto

Will you keep the contextual menu that allowed users to add charts, inputs, and many other things (which was very useful for students)? Or will we now have to ask the AI to do these things instead (which, in my opinion, is not a good idea)?

No, we’re not planning on reintroducing the “Add cell menu” as it was called. Many users found it disruptive and distracting when creating new cells. We’ll likely bring back equivalent code snippets that are available via code autocomplete, like these (although I find these snippets to be a little bit too opaque and want to find a way to see a little bit more of the code before it’s inserted):

Another technique is to have a notebook that includes code samples, and then copy-paste cells from the notebook (which also works between old and new Observable; select the cell with the checkmark in the notebook toolbar or by hitting Esc, then Mod-C to copy), or fork the notebook.

But yes, we do believe that canned snippets are extremely limited compared to what AI can do, and so are putting more effort into AI. The AI tends to do much better because it is more open-ended and can adapt to the context of the notebook. And we like to remove underused and underpowered features so that we can reduce the surface area of what we need to support and what users need to learn, though naturally we cannot satisfy the needs of everyone.

@neocarto

Will you bring back the “Embed” menu as it was before?

I don’t know if the menu will be exactly as it was before but absolutely, yes, we plan on supporting embeds, likely both iframe embeds and JavaScript imports (“runtime embeds”). The latter will likely be a paid feature, though you can always freely download your notebooks and self-host them (or develop them locally in the first place), and in many cases self-hosting is a good idea for performance and autonomy anyway.

[mbostock]

@jwb

Data table cells provided a quick and easy way to attach a new file, flip between different files, judge data quality at a glance, and do common column transformations without a code cell.

We hear you. Hopefully the forthcoming improvements to SQL cells will serve this purpose. If we make it easy enough to create a SQL cell from a file (something like SELECT * FROM file), then you’ll get the same visual summaries and filters as the past data table cell. But unlike the data table cell, the query can be much more expressive if and when you want it to be.

[mbostock]

@rusosnith

What should we do with “previously working notebooks that don’t work anymore” situations?

Yes, you can either let us know here in the forum if you don’t mind sharing it publicly, or you can email us at support@observablehq.com or by clicking the Email support link in the help menu. Thanks Andres and sorry for breaking your notebooks!