Announcing Observable 2.0!

Today we’re launching Observable 2.0 with a bold new vision: an open-source static site generator for building fast, beautiful data apps, dashboards, and reports. Read the announcement from founder Mike Bostock.

The cornerstone of Observable 2.0 is Observable Framework.

With Framework, you can:

  • Write data loaders in whatever language you want (Python, R, SQL, JavaScript, or anything else) to pre-prep data snapshots for fast page loads. Framework projects are just local files, so can easily integrate into your existing tools and workflows.
  • Use built-in themes and components for the fastest path to beautiful data displays — or write your own code for limitless customization.
  • Write vanilla JavaScript with Observable’s reactive runtime for flexible and fluid development.
  • Deploy anywhere. Or, deploy to Observable with one command.

And, Framework pairs perfectly with Observable notebooks for rapid, collaborative data exploration and prototyping.

Get started with Framework today:

We encourage you to share what and how you’re building with Observable Framework in Show and Tell!

20 Likes

This is really nice and I welcome the new direction!

I do have one question - doesn’t this obsolete your paywall model?

Yes. Observable is now free again for individuals, including private notebooks and databases connectors. And you can pay for larger (>5) teams and more hosted projects.

Our business focus is on productionizing (i.e., hosting and compute) data apps for teams. Observable Framework is open-source, so it’s free for you to do whatever you like with it. But if you’re building data apps professionally, our hope is that we offer a compelling complementary paid service for you to host and maintain your (often private) data apps on Observable. We make it easy for you to share data apps securely with your team or customers or clients or whoever, and we manage the complexities of keeping your app & data up-to-date with continuous deployment, scheduled builds, access control, collaboration, monitoring, analytics, etc. This starts with observable deploy today and will get better over time.

(And we think notebooks are a nice complement to Framework projects too, so you can quickly try out ideas.)

12 Likes

Congrats! I skimmed through the Framework documentation and examples and it looks very nice! Thank you for making this available for free!

Being a big fan of Observable Inputs/Plot, I have experimented with creating similar dashboards with quarto:

From a first look, it seem like these 2 frameworks (Framework and quarto dashboards) overlap a lot, from the perspective of creating interactive dashboards. What do you think are the main advantages for using Framework?

ps: very excited to start creating something with Framework :smile:

Alleluia ! Alleluia ! :pray:
This is a FANTASTIC news !!!

I just read the announcement, but given the quality of (better: the intelligence in) d3, Observable, Plot, I have zero doubt Framework will impress me juste the same !
Will read the doc very carefully.

It turns out I “agree”: I had been thinking: too bad the fantastic table ‘component’ could not be used outside notebooks; beyond a certain size, a notebook was becoming cumbersome; the export mechanism was a bit contrived.
Going back to a ‘regular’ dev experience feels like breaking free, and does actually free the mind :butterfly:

Just one problem:

I find the d3, Observable, Plot initiatives so good that I wanted to pay, by principle, and because beyond a few leisure notebooks I can only use these high quality tools inside thick corporate walls…

Solemnly, thank you and all the people working on this endeavour
:bowing_man:

3 Likes

I just played with and deployed the demo project, and it worked like a charm.
Framework is SO GOOD, and I can see it drastically speeding up my front dev.

From a business perspective, custom embedding of these reports / dashboards in my own application is a key use case. Will you develop more sharing features?
Some example ideas:

  • Expanding the sharing capabilities to non-users (email or SSO)
  • Using custom domains
  • Sharing parts from the “standalone” dash/report only. For example, being able to only embed one page or card in my app, as opposed to needing to load the whole project and then hiding the non-needed parts.

Thanks for the great work in any case, really excited to build on top of this!

1 Like

A follow-up on the self-deployment; the documentation says:

The build command generates the dist directory; you can then copy this directory to your static site server or preferred hosting service. To preview your built site locally, you can use a local static HTTP server such as http-server

With http-server the website renders perfectly fine; however I failed to get it running on Github Pages. I took the “dist” folder and cloned it in a Github repo and served it as Pages; however all styling and interactivity were stripped off, basically rendering the basic HTML.

I guess the deployment indeed needs a server, and can’t be simply rendered as a static page? Or am I missing something here?

That is definitely true. You can’t open an HTML file generated by Observable Framework directly in a webpage and expect it to work, for example.

I don’t know anything about GitHub pages but there must be a server for github.io - right? So, I wouldn’t think that’s the problem.

Yup. It needs an http server, but does not need an app server.
I guess more precise would be that you need a static web server instead of a dynamic web server as described here.

FWIW it worked as expected on GH Pages for me:

Built Site: Observable Framework

Repo (with GH Action to build and publish sources on push): GitHub - GordonSmith/framework: A static site generator for data apps, dashboards, reports, and more

Oh that’s interesting! Not sure why it didn’t work for me, will check your repo tomorrow - thanks!

FWIW It worked well when deployed as static site on Digital ocean; the only issue I saw was that the links between pages need to include a “.html” for the pages to be loaded correctly.

1 Like

OK using Github actions made the trick - thanks so much @GordonSmith !

This is great! It will simplify a process we have been putting in place to create dashboards. I have created a variety of Observable Notebooks using plot - are they compatible with the framework - i.e. exporting…? I want to get my dev team looking at the framework so I can focus on the data, analysis, and visualization.

The Framework CLI supports a convert command for downloading an Observable notebook and converting it to Markdown, e.g.,

observable convert @d3/bar-chart

will download the notebook and save a bar-chart.md and alphabet.csv to your current working directory. (We did it from the command line so it’s easy for you to automate or batch-convert or refresh and keep things in sync.) You may have to make some tweaks to the code due to Framework’s vanilla JavaScript syntax, but we’ll work on making that more seamless over time.

3 Likes

Hi @erdirstats! I’m a big fan of both Quarto (I still use it for my personal website, and as a “lab notebook” for data exploration, modeling, & stats projects) and Observable Framework!

A few things I am especially excited about in Observable Framework:

  • Data loaders (in R, Python, or anything else!) to pre-prep data snapshots for fast page loads. This was a bit of an adjustment at first, but now it makes my pages faster and my dashboards better, because I’m more deliberate about what a viewer needs to see
  • Framework pages are beautiful out of the box, and the built-in themes are :fire:
  • The grid & card css classes actually make custom dashboard layouts feel fun. This is not something I had experienced before :joy:
  • It’s really fast to get started with Framework. If you get a chance, check out the Getting started guide.

We’ll look forward to hearing about your experience & seeing what you build with Framework!

@mbostock When I copy and run your observable create command, I get the following error:

observable: unknown command 'convert'. See 'observable help'.

I tried to ensure that my observable command line is up to date by running

npm update observable 

but that did not fix the problem.

It’s because you have an old version of Framework installed. You need to upgrade @observablehq/framework not observable (the package name not the command name). You may also need to upgrade your global install depending on how you’re running it. (And if you’re using the Early Access version we gave you, you may have to edit your package.json to point to the ^1.0.0 release on npm.)

Yes, I’m sure you’re right. Note that the command observable version returns 0.0.50. I can’t seem to figure out the magic incantation to update it,though. I’ve tried

npm upgrade @observablehq/framework

and

sudo npm upgrade @observablehq/framework -g

Both commands appeared to install something but neither has had the desired effect.

This doesn’t make sense to me. I would think that you would run the observable convert command to create a new project based on a notebook. Thus, there wouldn’t be a package.json file yet. Nonetheless, I did try to

observable create

to start a new package and then ran an observable convert command from within there. Still no luck.

@mcmcclur I get confused by npm, so I’m not the best guide to why it goes awry, but I can at least try to help you get it to work! I was getting the same error, but yarn global upgrade @observablehq/framework worked for me. (I installed it with yarn, not npm.) Try which observable (to see where the version that runs is coming from), or npm list -g or yarn global list (to see what versions of @observablehq/create and @observablehq/framework you may have installed globally). You might also have a version of the old pre-release @observablehq/cli lying around, which could be confusing things.

In general, I find it easier to manage packages per-project. To create a new project, I run npm init @observablehq instead of the global observable create. If I want to ensure I’m creating a project with the latest version of Framework, I clear the cache with rm -rf ~/.npm/_npx and then run npm init @observablehq.

Inside a project, to use the local version of the observable command, you can run npm run observable convert @d3/bar-chart. (But that always seems to write the md to the project root for me, even if my working directory is /docs.)

I would think that you would run the observable convert command to create a new project based on a notebook.

A notebook is more analogous to one Markdown page within a project, not a whole project itself, so convert outputs a .md file. A project is more like a folder of notebooks, so creating a new project should be much more rare than creating a new notebook.

For that matter, a Framework page may not even be that analogous to a notebook. To riff a little on the “cultural” differences between notebooks and Framework projects: as we’ve acclimated to the strengths of each medium, we’ve come to think of notebooks as more like ephemeral open-ended scratchpads, and pages as more stable productionized artifacts. That means different things for different people and different notebooks, but, when we convert an internal notebook analyzing some business metrics to a page in our BI project, we don’t just adjust the syntax. We rewrite dynamic queries as more static data loaders optimized for the most important cases. We rethink the layout to be more of a dense 2D dashboard and less of a literary report. And we may condense several different notebooks exploring different aspects into a single project page, which takes up more “space” because it has to live in the same global project navigation.

To me this learning experience is a lot like learning Plot after D3. At first I try to recreate my favorite D3 things in Plot, and struggle with how that seems to go against the grain; then, gradually, I learn Plot’s strengths, and start making things I would never have tried with D3. And I also grow more comfortable with just sticking with D3 for the things it works better for!

4 Likes