Audio Worklet example

Show and Tell
1

It’s possible to get an audio worklet running nicely in Observable, but it takes a few workarounds. Here’s how I did it:

This might be a little nicer if the Observable editor somehow supported cells containing an embedded script, instead of a bare string.

Also, the trick of returning a Promise that never resolves to prevent downstream cells from running early seems a bit obscure. Maybe there’s a more elegant way to do it?

Edit: republished after I found a better way to convert a string to a URL.

3 Likes
2

Awesome that’s cool!

Yeah we need a better way for embedded scripts. I had a similar situation popup when making a Service Worker. I tried wrapping the raw script in a function, then toString the function and chopping the function framing off with substring. You can keep syntax highlighting then. See Offline embedded notebooks / Tom Larkworthy / Observable does it for a service worker. Not ideal though, and very fragile.

I throw off unresolved promises sometimes. Actually there are a few promise helpers in Promises.XXXX I think a Promises.never() would be a good additional. OK I made a PR as it seems like the right thing to me Add promises.never as alternative to returning an unresolved promise. by tomlarkworthy · Pull Request #218 · observablehq/stdlib · GitHub

1 Like
3

Mike Bostock suggested using the invalidation promise instead of a never resolved one which is much less dangerous, and seems to fit our use case of holding up a dataflow indefinitely.

The nice thing with the invalidation promise is that it will resolve if their is an upstream dependant change or code change, but of course you can keep returning it to hold off downstream indefinitely. So it kinda perfectly fits the gap we are plugging, and you can’t really accidentally cause a permenant resource leak.

4

I’m curious if this wouldn’t be a good use case for this on Observable? where it stores the cell’s previously returned value.

audioContext = {
  let ctx = this || {}
  if(ctx.constructor.name === "AudioContext") return ctx

  ctx = new (window.AudioContext || window.webkitAudioContext)();
  invalidation.then(() => ctx.close());
  
  await ctx.audioWorklet.addModule(scriptUrl);
  return ctx;
}

more about this

5

Interesting idea, but it seems tricky to combine with invalidation. We need to make sure that a closed AudioContext is never reused.

I suppose we could just never close it? But there are cases when we do want to recreate the AudioContext, like after editing the code in the worklet.

1 Like
6

Instead of a mutable you can use a coroutine-like yield the power of js generators / Anjana Vakil / Observable which is quite good for sequential stuff.

7

I’m not sure how I would do that. The tricky bit is how to avoid yielding a new value for running when volume changes in a way that doesn’t change the value of running.

I tried this:

running = {
  if (this) {
    console.log("not changing running");
    return invalidation; // no changes once the previous value is true
  }
  let result = volume > 0;
  console.log("changing running to $result");
  return result;
}

However, it seems that returning invalidation causes the downstream audioContext to be closed without creating a new one. This isn’t quite the same as not yielding anything at all.

8

here is how I replace the mutable with a yield (Audio Worklet Example / Tom Larkworthy / Observable) , with this construction you do not need to return invalidation promises, though you need to listen to the input change without going through dataflow so the cell does not reevaluate (ie. addEventListener to the viewof not the data stream)

I used to use mutables a fair amount but now I think I prefer this pattern as it avoid the ‘global’ though it does require more background knowledge to understand.

1 Like
9

Interesting. The behavior is a little different, though. It works when you run the demo as-is, but for development, if you change the worklet code then it stops playing (due to invalidation cleanup) and doesn’t automatically restart. You have to wiggle the volume control to get it to go again.

10

For the specific case of audio context you can test if it is running (Autoplay Policy Changes  |  Web  |  Google Developers)

So I fixed the example to provide the same behaviour by only awaiting for the input slider IF the context is not running. Thus if the cell is recomputed it skips the await.

1 Like
11

perfect! this is a much better way to do it!

12

Good find. I updated my example with this new approach.