Vega Lite Web Workers

Community Help
1

Is there a way to make Vega Lite work with web workers?

I tried something like that:
Vega Lite Web Workers

I found the basic idea here:
An update to the worker utility

Thank you very much!!!

2

I don’t think so, at least not in a straight-forward way. Observable’s vl is a wrapper around multiple liibraries:

and importing that directly from stdlib will fail because the packages get loaded via d3-require, which expects a document to attach the <script> elements to:

bar_chart = worker`
  return (async()=>{
    const vl = await import("https://esm.run/@observablehq/stdlib")
      .then(m => new m.Library())
      .then(l => l.vl())
    return (${plot_data})(${olympians});
  })()
`
1 Like
3

Thank you very much!!!

Is it possible with another library, e.g. d3 or Plot?
Or which library is the best fit for scalability?
For multiple plots like in this example:
Visualizing Pairwise Feature Interactions in Neural Additive Models
Just with bigger datasets, like MIMIC-III or California Housing, so that the Main Thread doesn’t get blocked.

Thank you!

4

It usually requires a lot of work to noticeably block the main thread. In my experience you’re more likely to run into rendering performance issues with complex SVGs (unless you’re doing some heavy computation), so vega-lite may have an edge here because the Observable wrapper defaults to the canvas renderer.

If your browser starts to lag you can wrap the rendering of each Plot or chart in requestAnimationFrame() which should give the browser UI enough room to catch up.

In Observable notebooks you can also await each cell’s visibility() promise which will only resolve once the cell is within the browser’s viewport:

myChart = (await visibility()), renderSomeChart()

or

myChart = {
  await visibility();
  return renderSomeChart();
}

For interactive charts that rerender on input changes you can debounce those updates if they happen too frequently.

In summary, I wouldn’t worry about it unless you’re starting to see real issues.

1 Like
5

Thank you!!!

I guess the main problem in my case is that all vega-lite plots get precomputed and are then displayed at once in a html-container.
I would have to rewrite the code, in a sense, that each plot is rendered separately, once finished.

6

Do you have a notebook that you can share?

7

Sure, if I load the California Housing dataset instead of the Penguins dataset, the whole notebook freezes for a minute.
Notebook:
Feature Interactions

Thank you!!!

8

The first thing that you’ll want to change is to get rid of any document queries. In Observable you can name your cells and then reference HTML from variable names:

plotsContainer = html`<div id="plots-container"></div>`

{
  const barChartContainers = plotsContainer.querySelectorAll(
    ".bar-chart-container"
  );
  // ....

The next thing to replace are side effect cells. In general you’ll want to ensure that cells only affect their own output or the output of cells that depend on them, otherwise you’ll run into dependency issues during imports, are prone to attaching event handlers that never get removed, and it becomes more difficult to reason about the dataflow in your notebook.

To ease the strain on the browser you can add a function

nextFrame = new Promise(resolve => requestAnimationFrame(resolve))

and then await it in precomputePlots():

      // ...
      const { key, dimension } = featureMap;
      await nextFrame();
      const plot =
      // ...

The downside here of course is that it will still take forever before the results become visible. So an alternative might be turn precomputePlaceholders into a generator function and let Observable process the generator for you (which also happens via requestAnimationFrame :slight_smile:):

async function* precomputePlots(originalData, featureMaps, minMaxPerClass) {
   // ...
      precomputedPlots[className][key] = plot;
      yield precomputedPlots;
   // ...

Note however that every yield will cause downstream cells to invalidate, so you’ll want to make sure that there’s not much happening down the line. In my tests the sandbox iframe frequently crashed, likely due to GPU related issues.

1 Like
9

Thank you very much! The generator hint works well in first tests.