Dataflow, a self-hosted Observable Notebook Editor

Show and Tell
1

yo! Wanted to share something I’ve been working on for a while, Dataflow, a self-hosted Observable notebook editor. It’s got support for FileAttachments, Secrets, custom standard libraries, importing local files and observablehq notebooks, and integrates with any text editor that you want! Observable notebooks in Dataflow are files on your computer, so it works very well with git.

The project is still pretty young, but feel free to try it out and share your feedback!

14 Likes
2

Congrats!

I’ve tried it and it has incredible completeness. Most of things works well, except Arquero’s table view command, like: Arquero Cookbook / UW Interactive Data Lab / Observable

the tables renders as html codes rather than elements in ‘Dataflow’. How to correct it?

3

thanks for trying it out!

What’s happening here is that aq is defined in this notebook, where the .table() view command is defined in the toView cell at the way bottom. But, that notebook uses the old html tagged template literal to define how to render the table, and while Dataflow uses the new html tagged template from htl (which is backwards incompatible).

As a workaround, you can fetch the old html tagged template and inject it into the import statement like so:

// workaround.ojs
oldHTML = new (await require("@observablehq/stdlib")).Library().html()


import { aq, op } with {oldHTML as html} from 'https://observablehq.com/@uwdata/arquero'

aq.table({
    country: ['USA', 'USA', 'Canada', 'Canada'],
    medal: ['gold', 'silver', 'gold', 'silver'],
    count: [10, 20, 7, 26]
  })
  .groupby('country') 
  .pivot('medal', 'count') 
  .view()

dataflow run workaround.ojs

image
image1838×606 38.2 KB

But a better longterm solution would be to fork and edit that notebook to use the new htl.html so it plays better by default with Dataflow. I’ll consider making Dataflow more backwards compatible with notebooks that use the old html template as well!

1 Like
4

Thank you for this! I’ve been wanting to try working with Observable notebooks in an IDE like VS Code for a variety of reasons: faster search, find and replace, refactoring (recursive renaming), etc. I’m most interested in experimenting with Deno (or Node.js) notebooks, Typescript notebooks could be great, too.

1 Like
5

Am really loving using this for a notebook I wanted to do with some very private very large data and it works super well!

I put in a PR to upgrade to the latest version of the observable standard library—would love if you’d take a look.

1 Like