Hi. I have an Observable Framework dashboard that works perfect when using observable preview (thanks for such a wonderful framework!!!)
But once I deploy (in my case to Observablehq), the same dashboard throw errors on the Frontend “RuntimeError: Importing a module script failed.”.
In the javascript console there are tons of errors “Failed to load resource: the server responded with a status of 404 ()”. Specifically, it seems to try to load a file .../_npm/d3-dsv@3.0.1/_esm.js (where the prefix would be the deploy path. Indeed, checking the dist directory and navigating to the _npm subdirectory, that module isn’t there.
I am puzzled that the preview works fine, but not the deployed product. Any help would be appreciated.
Some additional information would be helpful — if you can share the code to reproduce the bug, that would be especially helpful.
How are you loading d3-dsv? Are you importing it yourself, or are you using it indirectly, say by using d3 that is available by default in Markdown, or by using FileAttachment.csv? If the latter, can you share a snippet of the code you’re using to load the CSV file?
Framework uses static analysis to figure out which libraries you’re using, and therefore which files need to be self-hosted from npm and baked into your built site. But given the dynamic nature of JavaScript, it’s possible (though hopefully uncommon) to import a library that’s not detectable through static analysis. If you can share more of your code, we may be able to determine if this is the root cause.
If it is, you should be able to add an explicit import of d3-dsv to ensure that it’s included in your built site. For example:
import "npm:d3-dsv";
(If the file is present in your dist directory but doesn’t appear when accessed through observablehq.cloud, that would be a different bug. But it sounds like from your description that it’s a problem with building rather than with serving.)
I don’t explicitly use d3-dsv, but I am indeed probably using it under the hood for FileAttachment, but I’m not able to see how and where. I tested it with the explicit import, and that at least made that error go away, but I get lots of other errors now, about missing tables from the DuckDB database (the dashboard version that is deployed right now shows those).
I guess I’m puzzled about the fact that preview works flawless, but not the deploy. If there is a missing library in dist/_npm, or if the DuckDB db has a table missing, why doesn’t that affect also preview?
I don’t see any errors in the dashboard now. Maybe that was a temporary caching problem after you deployed? Do you still see the errors?
The reason the d3-dsv error doesn’t occur in preview is that the Preview server does not enforce that a requested library has an associated statically analyzable import somewhere in the code. The Preview server receives a request for /_npm/d3-dsv@3.0.1/_esm.js (or whatever) from the browser and simply loads the requested library as needed to populate the cache. Framework can’t take the same lazy approach when building the site; it needs to include the imported libraries greedily in the generated dist. And if the import isn’t statically analyzable, then Framework won’t know to include it.
It’d be nice if the Preview server only served imports that are statically analyzable so that this sort of error was visible during preview, but that’s harder to do (also you’re editing the code during preview, so the set of statically-referenced imports is constantly changing). Perhaps in the future…
As to why Framework can’t find the d3-dsv reference naturally, it’s because the CSV (or TSV) file is loaded through a ZIP file. The code looks like this:
Unfortunately, Framework can only detect an implicit import of d3-dsv when you call FileAttachment("foo.tsv").tsv() directly; it isn’t smart enough to do type analysis and know that archive is a ZipArchive, and that archive.file(…).tsv() requires d3-dsv under the hood. (And we don’t want to load d3-dsv in all cases, since there’s no guarantee you need it if you’re using FileAttachment.zip.)
So the best thing to do here would be to add an explicit import to eager.js:
import "npm:d3-dsv";
This way Framework will include the library, and it’s a good practice to have the import live in the same file as the code that needs it.
Alternatively, perhaps you could avoid the ZIP archive entirely, and just load the files directly. Framework supports archives so you can load files from within ZIP files directly without needing to process the ZIP file on the client.
Yes, unfortunately I do still see errors in the deployed dashboard, right on the landing (index) page. It persists different browsers and emptying cache, so to me it doesn’t look like a temporary thing. Can you still not see it? I attach a screenshot
OK, thanks for explaining all this regarding the zip-intermediate. Makes sense. I think I know how to make this data-loading more straight forward, and I’m working on it now. Using the direct-from-archive read looks really cool (didn’t know that), but not an option in my case, as I have dozens of files in that zip that I’m concatenating together. I will change the strategy and prepare this in the data loader itself, so it will just result in a single TSV, I think that’s simpler. Maybe that will also solve the issue with the missing tables that I currently still see.
Oh boy. Now it works for me too. No idea what that was. I tried with Chrome, Firefox and Safari, and on two different computers, and removed cache, and always showed the error… Oh well. Thanks for checking, though! I think I’m gonna mark this as solved now, as the import statement solved the original problem. Thanks to both of you!
