Ok, it seems I also forgot to remove import.meta.url, as described in how do i embed the notebook in react.
I was hoping the instructions in the embed window about file attachments working with babel would have fixed this. Is this work around still necessary or is there a fix?