Imported function (Promise) is not resolved until page is refreshed

Any ideas on how to avoid “RuntimeError: load could not be resolved” in the workbook at https://beta.observablehq.com/d/68d39176d77c034e ? The RuntimeError appears to occur only when the page is first loaded, but the problem occurs pretty consistently when first loading the page. Refreshing the page in the browser a few times (usually just once) seems to clear it up, but I thought that Promises were supposed to handle this sort of issue under the hood. Is load being imported and used correctly on this page? I’m very much a newbie at Observable. How can the notebook be modified to avoid the RuntimeError when importing the load function?

Thanks

1 Like

Can you click “Share link” in the notebook menu so we can see your notebook?

Done. Sorry I overlooked that part.

Thanks Mike. It’s shared now.

I’m not able to reproduce the error, but I do notice that the load function you import from @bumbeishvili/fetcher is dependent on the public demo instance of CORS Anywhere (cors-anywhere.herokuapp.com), which has no uptime guarantees and other documented caveats:

This server is only provided so that you can easily and quickly try out CORS Anywhere. To ensure that the service stays available to everyone, the number of requests per period is limited, except for requests from some explicitly whitelisted origins.

If you expect lots of traffic, please host your own instance of CORS Anywhere, and make sure that the CORS Anywhere server only whitelists your site to prevent others from using your instance of CORS Anywhere as an open proxy.

I recommend hosting the omnitruncated-24cell-wedge-rendered.json on GitHub Gist or some other CORS-accessible server as described in Introduction to Data.

2 Likes

Thanks Mike.

I had a feeling it might be related to the fact that the load function is using a CORS proxy, but if that were the only problem, I’d expect the load function to be resolved and only omnitruncated_24_cell would have a run time error when load is invoked unsuccessfully. The fact that the load function itself can’t be resolved made me wonder about the failed import since the second error is a natural consequence of the first one.

Does my line of thinking make sense about the load function itself failing to resolve?

Here’s a screen shot. I have confirmed that I’m not the only one having this same problem with this notebook.

I’ll look into getting the json file moved to a CORS enabled site like GitHub or possibly enabling CORS on the site where it’s currently hosted and see if that makes any difference.

Any you able to reproduce any errors reliably if you visit the fetcher notebook directly, or does it only occur on import?

It seems to only occur on import, and only about half the time. Reloading the page once or twice fixes it. As far as I can tell no javascript console messages show up differently when it succeeds vs. fails.

I’ve had other imports fail occasionally on other notebooks for no obvious reason, so it might not be related to this one specifically.

I haven’t ever had any trouble opening the fetcher notebook directly, but in that case, the function isn’t being invoked. The import error is pretty consistent for me though. I don’t know that I’ve ever loaded the page without an initial error, but refreshing once nearly always fixes it. Only on rare occasions do I have to refresh 2 or 3 times.

I’ve only messed with a couple of observable notebooks so far, so I can’t speak to how often other imports might fail as @jrus mentioned, though I don’t recall any others with similar problems.

As I mentioned, I don’t have much experience with Observable, but one thing does appear odd to me within the load function. The initial attempt (roughly) calls getData(url,type).then(resolve(result)), but when that fails, the catch block calls resolve(getData(corsUrl, type)). It seems odd to me that the calling order is reversed within the catch block. Is there some reason why the author wouldn’t use the same code for the original try and then repeat it again within the catch block, but substituting the cors url the second time? It’s probably not related to my stated problem, but it just seems a bit odd to me. Any insights, or is this unrelated?

I forked the original page and made this new version at https://beta.observablehq.com/d/dfc7632b77475d9f. The original is unchanged in case anyone wants to compare its behavior. The only change to this new one is that it loads the json file from https://gist.githubusercontent.com. As I understand it, that means that CORS isn’t necessary for the load function, although I have no way of telling if the CORS proxy is actually being used within the catch block.

This one has the same erroneous behavior as I originally reported. Initially, the load function is not resolved, then one refresh fixes everything.

Here’s a screen shot of the new page showing the same errors:

One more detail I noticed… If I load the page once, then refresh it until it works correctly, then let it sit for some time, say maybe 15 to 30 minutes, then refresh it again, it exhibits that same problem as when it’s first loaded. Refreshing again right away resolves everything again. Any chance this is some sort of caching issue or maybe a load sharing problem in a web server farm?

Could the “load could not be resolved” be a red herring? Perhaps the import is working, but when the load implementation fails within the resolved promise, the error is mis-attributed as a failure of the promise itself… or something like that.

I’m more likely to trust the implementation of “import” than I am of that “load” function, though I’m sure that import has hidden complexities.

Hi @david-hall

I am the author of imported fetcher notebook

So, I visited your notebook and it works well for me …

There are some interesting discussions here, so I will try to answer some of the things

  1. Fetcher notebook uses cors-anywhere server when a direct call to the resource fails.
    So, after call failure, the code tries to load the same resource using cors URL

    getData(corsBypasserUrl+url, type),

  2. I can’t figure out, what you mean by

    It seems odd to me that the calling order is reversed within the catch block

    because it does not look reversed for me, can you explain it to me more?

  3. As Mike noted, the cors-anywhere server is only provided for the demonstration and trial purposes
    So I have added the notice inside fetcher notebook

    You can see it already

P.S. I am open for improvements and suggestions, so if you will have some ideas, it would be great

Edit 2: My idea about the problem was obviously wrong, see the Mike’s answer

Edit: Also, I think I have figured your problem out:

it looks like in some cases, a proxy server is also unable to access the requested resource, I tried accessing it from the proxy and got

Which happens when resource server is not accepting proxy server’s request

See more here


The advantage of hosting your data on a CORS-accessible server (such as GitHub Gist) is that you don’t need the load function and the corresponding import and dependency on CORS-Anywhere.

You can just use fetch:

data = fetch(url).then(response => response.json())
3 Likes

I am using load function mostly for the real-time information loading, when I don’t have control over the data source and data changes often, such as currency rate, it bypasses most of the barriers very fast without much hassle
(http <=> https, cors , google drive api)

1 Like

I was lucky enough to reproduce this error (in dfc7632b77475d9f) and after visualizing the notebook graph, was able to determine that the error was in requiring the Tabletop dependency. This breaks load because of the dependency loadgetDataloadGoogleSpreadsheetTabletop. I’ve not yet determined why Tabletop fails to load, however.

Ah ha! Okay, I’ve solved it.

The problem is that Tabletop fails to load if window.module.exports is truthy. This is… regrettably… a semi-standard UMD pattern. Here’s the relevant code from tabletop.js:

if (typeof module !== 'undefined' && module.exports) {
  module.exports = Tabletop;
} else if (typeof define === 'function' && define.amd) {
  define(function () {
    return Tabletop;
  });
} else {
  window.Tabletop = Tabletop;
}

This CommonJS export pattern is unintentionally triggered because of the imported @vorth/vzome-rendering-component, which sets window.module to load THREE.OrbitControls. As a result, the require of Tabletop fails if it happens to run after the loading of THREE.OrbitControls; the order is non-deterministic, so sometimes it succeeds and sometimes it fails.

That pattern was likely copied from my Hello, Three.js Orbit Controls notebook, so I’ve updated my notebook to delete the window.module after loading the Orbit Controls extension.

THREE = {
  const THREE = await require("three@0.82.1/build/three.min.js");
  if (!THREE.OrbitControls) { // If not previously loaded.
    const module = window.module = {exports: undefined};
    try {
      await require("three-orbit-controls@82.1.0/index.js");
    } catch (error) {
      THREE.OrbitControls = module.exports(THREE);
      delete window.module;
    }
  }
  return THREE;
}

This is a regrettable consequence of the long history of JavaScript’s lack of standardization around modules. Now that we have finally have an ES module standard, we can hopefully more towards better interoperability.

3 Likes

Wow, this is cool :thinking: , I just wish normal observablehq users had same dependency tree(network) visualization capability

They do!

(The only caveat is that to visualize someone else’s notebook, you currently have to create a fork of their notebook.) This restriction has been removed. Here’s a direct link to the visualization for @‍bumbeishvili/fetcher:

2 Likes

Thanks for tracking those down Mike!

Should folks be going upstream (e.g. to 3js and to Tabletop) and advising them to change their code? Or are there lingering reasons for them to keep doing what they have been?

Do you know of a good explanation somewhere of 2018 best practices for publishing a JS module so that it is usable by as many people as possible? Say for a brand new project hosted on Github that has code broken into multiple files, needs tests, and should ideally be usable either in a browser (e.g. observable notebook) or in nodejs.

(The last time I spent any time researching JS module stuff was 5+ years ago, and it seems to have changed / settled dramatically in the mean time.)

Maybe this should be its own discussion thread.