Possibility of canned datasets built-in?

Community Help
1

In some of my notebooks I want CPI-U so I can adjust everything for inflation. BLS does not provide CPI-U as a web service so I can’t just load it from a URL. I can just go to BLS and download it, of course, but it would be cool if that and some other key series were built into the product, with the data already in the right shape and format for ready use. Other examples of things that would be cool to have without needing to do it myself might be headline Census figures like populations of states.

1 Like
2

Have you considered creating a notebook to fetch and transform the data, and import it into other notebooks from there?

Observable’s example datasets all have in common that they never change, and a consumer price index sounds like something that you’d probably want to keep up to date?

As for the data source, is this the data that you’re referring to? https://www.bls.gov/news.release/cpi.t01.htm

I recently created a notebook to extract data from tables published by The Fed, and the tables in the page above seem to apply similar conventions: The Fed - FOMC Projections Data / Fabian Iwand | Observable

Here’s a self contained example, adapted from the notebook above:

// Adapted from https://observablehq.com/@mootari/fed-tidy-data
{
  const url = "https://www.bls.gov/news.release/cpi.t01.htm";
  
  const table = await fetch(`https://corsproxy.io?${encodeURIComponent(url)}`)
    .then(r => r.text())
    .then(t => new DOMParser().parseFromString(t, "text/html"))
    .then(dom => dom.querySelector("table"));
  
  const headers = n => {
    const headers = n.getAttribute("headers") ?? "";
    if(!headers.length) return [];
    return headers.split(" ").map(id => table.querySelector(`[id="${id}"]`));
  }
                               
  return [...table.querySelectorAll("td")].map(n => ({
    value: n.textContent,
    ...Object.fromEntries(
      headers(n).map((h, i) => [
        headers(h)?.[0]?.textContent ?? `dim_${i + 1}`,
        h.textContent
      ])
    )
  }))
  
}
1 Like