🏠 back to Observable

Fetch an XML document (Thredds Data Server catalog)

Is there a way to fetch an XML file (a catalog or a WMS getCapacities response) ?

I have started a prototype notebook with a Leaflet map that displays a WMS layer
distributed from a TDS (Thredds Data Server).
Have a look to: WMS GCA / Patrick Brockmann / Observable

Now, I would like to fill an inputs radio with the results from a request on what are the different layers queryable (available as an XML response from the TDS).

Any idea ?

In principle this should totally be possible!

Looks like there are at least two headaches with that one you’re trying, https://www.globalcarbonatlas.org:8443/thredds/catalog/catalog.xml:

  1. It doesn’t have CORS enabled, so you’d have to use a CORS proxy to fetch it; you can get around this, though it’s a bit of a pain: Setup your own CORS proxy / Sylvain Lesage / Observable.

  2. It demands https but its certificate is expired or invalid or something, so even if you get around CORS you get a security error. I’m not sure how to get around this!

Could you download the XML file and upload it to the notebook as a file attachment? That might be unsatisfying, sorry. But once you get past that, here’s how you can use an XML file to fill a list of radio buttons:

Hi,
Many thanks in you interest in the question.
I have just investigated the XML reading.
The response from my server should work. Accept the certificat
that I have declared by myself. If it is not working, I will rebuild a prototype
by using a demo TDS from Unidata. The Observable notebook might interest them
in addition to people from Reading that have developped ncWMS2.

So for now, the different queryable layers cannot be found. Perhaps a limitation
of xmlToJSON.

Have a look to:

looks like that xml is failing to parse correctly because it has ampersands in URLs. i escaped the ampersands (just find an replace all to &) and it now seems to work – i can get the expected array:

added to bottom of XML to radio button options / Toph Tucker / Observable

Indeed, those ampersands were the cause of the trouble.
I do not understand why the TDS delivers this kind of XML
since & is a special character.

Other TDServers give the same XML.
https://wms.hycom.org/thredds/wms/GLBu0.08/expt_93.0/ssh?service=WMS&version=1.3.0&request=GetCapabilities

But a replaceAll() fixes this easily. :grinning:

Now next step is to fetch it directly. I have used an external TDS (not mine) and
the response is not available.
What am I missing ?

I have added directives to web.xml in my tomcat 8.5

<filter>
  <filter-name>CorsFilter</filter-name>
  <filter-class>org.apache.catalina.filters.CorsFilter</filter-class>
  <init-param>
    <param-name>cors.allowed.origins</param-name>
    <param-value>*</param-value>
  </init-param>
</filter>
<filter-mapping>
  <filter-name>CorsFilter</filter-name>
  <url-pattern>/*</url-pattern>
</filter-mapping>

and now the fetch is possible.
Really cool…

Another problem is how to get attributs from an XML response. Not nodes but attributs.

<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.2">\r
  <service name="all" serviceType="compound" base="">\r
    <service name="odap" serviceType="OPeNDAP" base="/thredds/dodsC/" />\r
    <service name="wcs" serviceType="WCS" base="/thredds/wcs/" />\r
    <service name="wms" serviceType="WMS" base="/thredds/wms/" />\r
    <service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/grid/" />\r
  </service>\r
...

How to get “odap”, “wcs”, “wms”, “ncss” ?

Like this, e.g.:

catalog = new DOMParser().parseFromString(catalogXML, 'text/xml')
Object.fromEntries(
  Array.from(
    catalog.querySelectorAll('service > service'),
    n => [
      n.getAttribute('name'),
      Object.fromEntries(Array.from(n.attributes, a => [a.name, a.value]))
    ]
  )
)

Or if you want to query a specific service:

catalog.querySelector('service[serviceType="WCS"]').getAttribute('base')

By the way, for https://www.globalcarbonatlas.org:8443 you may want to switch to Let’s Encrypt instead of using a self-signed certificate. Their system packages have various options for auto renewal.

Thanks mootari !
Your are right about the self-signed certificate. Just here for testing.
I will declare it properly soon.

Many thanks for you help. DOMParser is perfect to handle XML.
It is good to have a community with Observable.

Some questions I will investigate.

  • how to keep user choices from inputs since they are built dynamically with a default choice ?
  • how to keep zoom from the Leaflet map ?

You may want to start separate topics for these if you need outside feedback. :slight_smile:

OK you are right.
So to keep in mind the final Observable notebook is available from: