Best practices for embedding?

Community Help
1

Does anyone have recommendations on how to embed cells in a web page for fast loading and minimal impact on the observable servers if there are many page viewers?

I am currently using the cell embed “Runtime with Javascript” mode, but my page load takes ~9s, mostly dominated by 0.3-0.4s to load each FileAttachment and imported notebook. The import fetches are pipelined, but the 18 FileAttachments (small images) are fetched serially.

In case it’s relevant, I am benchmarking this page with cells embedded from DESI in 3D / David Kirkby / Observable.

2 Likes
2

I think the slowdown is that all the image file attachment promises have to resolve before there is any rendering. You could maybe use a sprite sheet to make the file loading faster, or you could implement a placeholder solution so they render as empty or black squares before the images are loaded. I sketched out a quick idea here if that helps: Placeholder Image While File Attachment Image Loads / Observable / Observable

2 Likes
3

Inside textures, the logic loops through every texture image sequentially to fetch. You can probably use Promise.all to do the fetching of all images concurrently to make that cell run faster (it current takes ~5s to resolve, the slowest cell by far).

But if you want to reduce the # of requests that get sent to Observable server, ^^ spritesheets sound like a good idea! Alternatively, you can host the file attachments yourself on your own server. If you choose the “download code” option on the notebook menu, the downloaded .tar.gz should have all your file attachments inside, which would make it easier (then you’ll just have to change the index.html file inside to embed the cells that you want).

2 Likes
4

Like @asg017 suggested, you can speed up and simplify your texture loading by rewriting it to this:

textures = {
  const loader = new THREE.TextureLoader();
  const load = image => new Promise(async resolve => loader.load(await image.url(), resolve));
  const entries = Object.entries(textureImages).map(async ([key, images]) => {
    const textures = await Promise.all(images.map(load));
    return [key, textures];
  });
  return Object.fromEntries(await Promise.all(entries));
}

Note: This could be code-golfed further, but I’ve opted for readability instead.

1 Like
5

There are obviously some solid recommendations and code for working within Observable already. For maximum speed and minimum variability, it would be hard to beat the “spritesheet” recommendation. The basic idea is to set up a webpage that holds pre-written copies of all the content with set dimensions at the start. I set that up for you notebook and you can see the result on this web page. Note, though, that I only implemented the content up until the “Play” button. It’s actually nice that way, since you can see the difference in loading.

One other comment, your observerText toggle buttons don’t work because you’ve got to embed the toggled cell as well, since it works via side effect. I included that cell but set its style to display: none so that it works in my demo.

6

Thank you all for the very helpful suggestions! I have quite a bit to learn here, but will be trying this all out soon.

7

I created a github issue to track my work on this.

The parallel fetches suggested by @asg017 and @mootari using Promise.all give more than a 2x speed up and @mcmcclur’s static initial page idea improves the user experience a lot. Implementing a sprite sheet needs more work but is no longer the bottleneck.

I have two followup questions:

  • Is there an obvious reason why all the 20-30K PNG fetches have time-to-first-byte ~300ms while the 270 JPG fetch is only ~45ms?
  • The third round of fetches are for nested imports that are not actually needed. For example, I import a notebook that imports vega-lite to demonstrate its functionality. I guess there is no way to avoid this without refactoring the imported notebook into separate API and examples notebooks (which somewhat defeats the huge appeal of observable)?
8

Yes: The JPEGs are hosted by Github, the PNGs by Observable. Edit: I only see a few JPEGs. What do I have to do to trigger the load of the bulk?

I guess there is no way to avoid this without refactoring the imported notebook

Indeed, notebook imports are static imports, which means that the whole file will be loaded. However vega-lite only gets loaded because it is a transitive dependency of one of the cells that you import. You could analyze your notebooks via Notebook visualizer / Observable | Observable to figure out what to cull or restructure.

9

Yes, its just github vs observable hosting: I moved all the images to github now.

I also refactored the imported notebooks to eliminate all the depth-3 fetches.

Now, the time is dominated by the observable TTFB ~ 0.4s to load a notebook .js. I suppose I can copy these also to github, but that requires some manual steps beyond clicking “Republish” so I will leave that until later.