I try to communicate with an API
With some it works fine
`book-url = "https://www.booknomads.com/api/v0/isbn/9789000010134"
books = (await fetch(book-url)).json()
// get the response`
but with other it does not
`ick-url = "https://iclikval.riken.jp/api/annotation"
annots = (await fetch(annotationURL)).json()
// error 403 The origin "null" is not authorized`
If I use the url in my browser the origin is not set, from a server the origin is set to “http://myserver”. In both case the get request works. But when use Observable, the origin is set to “null” as a string, that is not valid and is rejected by the API…
Browsers have security policies around XHR requests and the fetch() API that prevents websites from making unauthorized requests to other domains on the user’s behalf.
In order to make it possible for an external website (like Observable) to hit an API from the browser, the API needs to include a CORS header in the response — in this case:
Access-Control-Allow-Origin: *
The booknomads.com API includes that header in its response, and the iclickval API does not. That’s why the first works and the second fails.
I’m afraid there’s nothing we can do at the moment to help you work around this. You’ll have to set up a proxy on a server you control, or get iclickval to add the appropriate headers to their API. In the future, we hope to offer data proxying and authentication services as part of Observable, but that’s a ways off yet.
If the site that you are trying to load data from does not have CORS enabled, or only supports HTTP (not HTTPS), you might yet be able to load it via a CORS proxy such as CORS Anywhere.
For example, if your URL is:
https://google.com/robots.txt
Then you’ll get a TypeError: Failed to fetch if you try to load it:
data = d3.text("https://google.com/robots.txt")
To load it via CORS, prepend the cors-anywhere demo server domain to the URL:
data = d3.text("https://cors-anywhere.herokuapp.com/https://google.com/robots.txt")
However, you shouldn’t publish a notebook that uses CORS Anywhere to load data; only use it locally for testing purposes. Per the CORS anywhere documentation:
This server is only provided so that you can easily and quickly try out CORS Anywhere. To ensure that the service stays available to everyone, the number of requests per period is limited, except for requests from some explicitly whitelisted origins.
If you expect lots of traffic, please host your own instance of CORS Anywhere, and make sure that the CORS Anywhere server only whitelists your site to prevent others from using your instance of CORS Anywhere as an open proxy.
If your data is static, one option is to download it and host it on GitHub Gist as described in the Introduction to Data tutorial.
I contacted iclikval. Their answer is
" Origin: “null” is an invalid origin, and then get rejected by the server framework."
if Origin in absent of the header, the request pass.
if Origin is set to http://anyAddressIsOk, the request pass. (https also pass)
if Origin is set to “null”, the request is rejected.
another bit of info is when I use fetch in codepen, the Origin is set to https://s.codepen.io, that meet the scheme http:// + address and then pass…
I don’t know the specific about the standards,
if it is a server issue or a client issue…
but that is very troublesome :-/
Hi, this seems to have died, but it’s a problem that must be solved. I fully understand how CORS works but this in not solvable even when I have control of the server, because Observable is not sending a correct origin header. Like codepen, it should be sending an origin like “beta.observablehq.com”, not “null”. In some cases I can work around this but it makes it impossible to use socket.io, for example:
“The value of the ‘Access-Control-Allow-Origin’ header in the response must not be the wildcard ‘*’ when the request’s credentials mode is ‘include’. Origin ‘null’ is therefore not allowed access. The credentials mode of requests initiated by the XMLHttpRequest is controlled by the withCredentials attribute.”
I.e. Allows * as origin does not suffice for Chrome, I need observable to send the correct origin so that I can allow beta.observablehq.com explicitly on my server.
Yeah, I second that, this is REALLY annoying. We’ve been trying to work around this in multiple ways but the null thing makes it very, very, very hard. Can you please make sure that it’s a valid domain there?
We’re working on it! There are security (and DNS) considerations here, so we have to be careful to launch this correctly, but it is our plan to enable a non-null origin for notebooks in the near future.
I think the main issue with the way observable sets up origin for code cell is that unlike the notebook itself, which has https://beta.observablehq.com origin, code cell frames are pinned to custom user static content.
See origin request header generated from a fetch() cell call:
Therefore, even if you have control over the server hosting a data API you’d like others to use on observable, you can’t simply add beta.observable.hq.com to the ‘Access-Control-Allow-Origin’ whitelist response header for all devs on this site to use your API without CORS proxies, unless you * for the public, which is not recommended
In fact, you can’t even use observable api itself, despite the fact that notebook document makes those requests for user info, collections, etc.
I put a short fetch.js notebook together to demonstrate this here (with links to specs and other fetch notebooks):
Anyone can sign up for Observable, so enabling access to all of Observable is effectively the same as making it accessible to everyone. If this is what you want, then use:
Access-Control-Allow-Origin: *
If you want only specific users on Observable to have access to your site, then have your server check the Origin request header and conditionally return the appropriate Access-Control-Allow-Origin response header
https://${user}.static.observableusercontent.com
where ${user} is the login of person you wish to grant access. For example, my login is mbostock, so for me:
Because Observable provides separate hosts for each user, you can control which notebooks are able to communicate with your server. In conjunction with other forms of access control, such as cookies and network firewalls (intranets), you can have private servers that are accessible only to specific Observable users.
We should have a tutorial on this soon, but I wanted to clarify in this thread now.
I presume you are referring to the fact that we haven’t launched a public API for Observable? The choice to not make the current internal API public is wholly intentional; it’s not public (as in: you are not intended to use it). We intend to change the current API in the future in ways that would break current usage, and making it public now would falsely indicate a williness to support public usage, when our development efforts are focused on the platform itself.
nah, was not about that. I worked around that quickly, and can patch it later in a day if you change it.
you should add userKey back and give devs ways to explore your api with that.
let’s table it for now. we got this far and I love your beta! it’s the closest any notebooking sys got to winning my and that’s coming from a guy who comes from taiga
Basically, using ajax with local resources doesn’t work.
Chrome and Safari has a restriction on using ajax with local resources. This error means that you are trying to perform Ajax on a local file. This is forbidden for security reasons.
In order to solve this problem, you can use firefox or upload your data to a temporary server. If you still want to use Chrome, start it with the below option;
--allow-file-access-from-files
Also, this kind of trouble is now partially solved simply by using the following jQuery instruction:
Problem solved
and I love your beta! it’s the closest any notebooking sys got to winning my 
and that’s coming from a guy who comes from taiga