importCell conflicts with static import

Hi,

@mootari made me discover Dataflow / Mike Bostock / Observable which provides the importCell() function to dynamically import cells from another notebook.

I updated the small notebook where I showed how to do the same using the import {...} with {...} from '...' syntax.

But both examples collide, and to make them work, I’ve had to make the dynamic import depend on the static import, to be sure it’s evaluated after.

What is the reason? And is there a better way to use both in the same notebook?

1 Like

Could you try removing the workaround? After an initial failure, I can’t seem to replicate the problem anymore. (Perhaps a caching issue?)

Edit: Ah, there it is again, after removing and adding back the import in a new cell.
Edit 2: I can’t reliably reproduce the error. Still recommending to remove the workaround, for easier investigation.

1 Like

Oh, you’re right, it only occurs if the cache is deactivated, or if it’s the cache is empty (some request has still not been cached). Reloading the page with the cache activated removes the error.

So: we still need to understand why it fails on the first visit.

I removed the workaround

Could you republish your notebook without the workaround, please?

1 Like

I suspect that this is a conflict between two d3-require instances:

  1. packaged with Observable’s Runtime, loads @mbostock/dataflow#d3
  2. packaged with runtime.umd.js which is loaded in @mbostock/dataflow#Runtime; attempts to load @d3/histogram#d3

This second attempt causes the error. I think @mbostock may want to take a look at that bug.

2 Likes

Thanks for taking the time to dig into that problem. I wouldn’t have been able to reach that point.

1 Like

I haven’t had a chance to dig into the issue here, but I think I ran into something similar in this old thread:

2 Likes

The process was:

  1. verify that you can reliably reproduce the error (:+1: for that with cache disabled)
  2. in your developer tools, open the “Search” drawer and search for “invalid module” (it should appear in two files)
  3. prettify each file and set a breakpoint in the corresponding line (you may have to search the file for “invalid module”)
  4. reload, and wait for the breakpoint to hit

From here on we have two objectives:

  1. find out where the error originated
  2. find out the nature (and possibly the cause) of the error

The first can be done fairly easily by looking through the call stack, but finding the right place requires some intuition. In the screenshot below we find the notebook cell that the error originated from:

The second point on the other hand is a lot messier, especially when working with minified code. In order to orient yourself, you’ll want to have some unminified reference code at hand. Here is the one for d3-require.

By relating the minified code with the reference code, we find the line where we had set our breakpoint.

Now we jump back to the top of the call stack, collapse it, and take a closer look at the scope below it:

On the right side, we see the actual error message. By hovering over the variable “Ce” we can inspect its contents, and conclude that Ce.pop()returned undefinedbecause the array is empty. Looking the ref. source, we related Ce to queue.

From here on it gets somewhat handwavy. We can set additional breakpoints for Ce.push and Ce.pop in order to poke around in the values that pass through. Doing so (and inspecting both anonymous function bodies and surrounding scope), we discover that there’s another d3 instance which gets loaded successfully.

When you’ve messed around with d3-require in the past, you may have noticed that it caches resolved modules, and also sets some global variables (an unfortunate requirement of the AMD format).

Tip: It’s good to keep in mind that you can interact with the active breakpoint’s scope in the JS console.

2 Likes

Thanks a lot for the detailed answer, it’s very useful, and I could reproduce the process.

Do you know if the developer tools provide a way to search for a string (“invalid module” in our case) in all the sources, or is it necessary to open every file then search in it?

Yes, that is the “search drawer” to which I was referring in point 2:

  1. in your developer tools, open the “Search” drawer and search for “invalid module” (it should appear in two files)

The “drawer” is the section at the bottom with the JS console, that can be toggled (hence “drawer”). You can add the search tab via the dot menu to the left:

1 Like

OK, thanks! Firefox does not provide a Search tab :man_shrugging:. I will use Chromium the next time I have to do hardcore debugging

Firefox also lets you search across all files. If you’re on macOS though, make sure that you’re in the debugger panel. Otherwise you’ll just toggle fullscreen. :slight_smile:

https://developer.mozilla.org/en-US/docs/Tools/Debugger/How_to/Search#searching_in_all_files

1 Like

Thanks, it’s perfect

I made a tweak to my notebook so that importCell passes through the existing require function, rather than loading a new one.

// Instantiate a new runtime, but reuse the existing runtime’s require.
// (This is ensures that any requires in imports still work.)
const library = Object.assign(new Library(), {require: () => require});
const runtime = new Runtime(library);

That seems to have fixed the problem. Thanks for doing all the legwork!

6 Likes