Easy way to add processing power to observable notebook?

Community Help
1

I apologize if this is a dumb question: I have a notebook that is a pretty dense force-directed graph that is just making observable chug. I have easy access to powerful vms/servers: is there any easy way to “plug” observable into one of these so the processing is being done by something more powerful?

2

Good question; this comes up sometimes and we don’t have a great answer! No, there’s not really an easy official way to plug in more compute. The computation is all done locally, in your browser. So you’re in control of that, and you could run it on a more powerful server. If it helps with portability, you could download the notebook as a standalone package (⋯ → Export → Download code) and run that version.

@mootari has experimented with using Web Workers to speed up force-directed graphs in the browser: Interactive Force-Directed Web Worker / Fabian Iwand / Observable. And I think @tomlarkworthy has experimented with running notebooks “headless” on remote servers.

1 Like
3

Please be aware that the current version of that notebook doesn’t use workers (although it still has the worker code in it). For a version that actually uses workers see

2 Likes
4

The notebook that Toph linked uses ngraph in the main thread, and at the time I found the performance to be even superior to the webworker version with d3.

How many nodes and links are we talking about? Also, do you want to render a settled graph (i.e., with an alpha of zero) or do you still want to interact with it (like dragging, adding or removing nodes etc)?

5

Awesome: I have pretty easy access to graphical Linux VMs so I’ll start there. Question, though: do you know if the force-directed graphs are hitting GPU, CPU, both? We can customize the machines pretty easily so it’ll just cut out some experimentation on our end if we know how to spec.

6

Both ngraph and d3.force use the CPU. For the GPU you may want to give ParaGraphL a try (haven’t tested it myself yet, though).

Edit: Note that JavaScript is single-threaded, so anything that uses the CPU will only use a single core (unless you do multi-threading via webworkers).