Copy a new data file to a build of Observable Framework

Hi, due to specific use cases, I need to create a build of a framework project that can read a data file from the filesystem of a user. The build is distributed to users through a zip file. Please note that each user has her very own data file that she can copy in a specific path, but she can’t run a new npm build. Users can serve the static site locally with the http server bundled with Python, and then they can explore their data.

For the moment, I have the following function to load the data file:

async function loaddata() {
  const response = await fetch("_file/data/userDataFile.json");
  const data = await response.json();
  return data;
}

The data path is hardcoded and points to a directory of the static build. Each user is expected dt copy manually her data file into _file/data/ to make this machinery work. Is there a better alternative to this workflow?

If I understand your use case correctly, you should be able to deploy a Framework project (no need for users to run it locally) and have users provide a local file to the hosted page by using a File input. See the documentation here

1 Like

Ho @Cobus! Thanks for your suggestion. Unfortunately, I can’t follow that approach because users can’t upload their files to external services due to the privacy policy. That’s the reason why I’m trying to implement a different pattern as straightforward as possible.

When you use the File input, the file is not uploaded to a site, it is simply loaded into the user’s browser where your code can then read it, parse it and dynamically generate the data visualizations. The file never leaves the user’s computer.

Think of it as hosted code with local data.

1 Like

Thank you for the clarification. That could be a very good solution!

I was trying to automate the process as well, with a minimal manual intervention. Since such users are on Win, I wrote a simple .bat file, bundled with the project, which gets data from the network and creates the data file. Then, it copies such a file to _file/data/, and finally runs the local python server and opens Google Chrome pointing the browser to localhost. So: just one click to get the data viz. If I wanted to stick to this one-click process, I’m afraid I’d need to hardcode the file location, pointing to the build path. Quite ugly, since it doesn’t work in dev mode, unless I have such a path in my dev env.

Thank you!

1 Like

I think if you have the hard requirement that you can’t have a build process, or at least that you can’t run observable build in your build process, that this is as good as you’ll get. (Though there’s no obligation to put your data in _file/data — it could live anywhere convenient for you.)

If you were to allow observable build then you could implement this as a data loader instead that reads the local file and either copies it or transforms it. That would give you some additional flexibility to process the data at build time, too.

1 Like

Thank you, @mbostock and @Cobus, for your precious help. Data loaders have been my first choice, but unfortunately, users can’t install Node.js, so observable will not be available in their operating system.