Tensorflow.js bin file

Community Help
1

Hi,

I created a Tensorflow model in Python, then generated the model.json containing the architecture and a group1-shard1of1.bin file containing the weights. I attached them both to my notebook, then loaded the model with

tf = require('@tensorflow/tfjs@3.15.0')
model = await tf.loadLayersModel("model.json");

I got an error model = TypeError: NetworkError when attempting to fetch resource. I guessed that this was because the json file couldn’t find the bin file (by default, it looks for the bin file in the same folder as the one of the json file). Any idea how can the json file point to the correct path?

Thanks!

2

There are no file attachments in the notebook that you shared.

Once you’ve added the files to your notebook, you need to reference them via the FileAttachment helper. For example, to load the model, use:

model = await tf.loadLayersModel(await FileAttachment("model.json").url());

You can read more about FileAttachments here: File Attachments / Observable / Observable

3

Indeed, files were probably not published since they were not used. I’ve tried your code, but got another error (RangeError: buffer length for Float32Array should be a multiple of 4). So I tried to load the model in an async function, as I did in a local functioning prototype, but still got the same error. Maybe again because the json file points to the bin file at the wrong place. Any idea how to correct this?

Thanks!

4

If you take a look at the dev console or network tab you’ll notice that tensorflow still tries to fetch the binary from the wrong path, relative to the model path. That path is defined in the JSON (at model.weightsManifest[0].paths[0]), which makes it awkward to change. I’m not really familiar with tensorflow, but the following solution seems to work.

First, create a cell that lists all your file names and their URLs:

files = new Map(await Promise.all([
  FileAttachment("model.json"),
  FileAttachment("group1-shard1of1.bin")
].map(async f => [f.name, await f.url()])))

Then you load your model by overriding the fetch function:

model = await tf.loadLayersModel("/model.json", {
  fetchFunc: (url, ...options) => fetch(files.get(url.slice(1)), ...options)
})
1 Like
5

Wow, it works great! Thanks

6

Awesome, you’re welcome! Note that you can mark the topic as answered by clicking the “Solution” button under a post.

1 Like