# Good data hosting options?

**URL:** <https://talk.observablehq.com/t/good-data-hosting-options/5235>\
**Category:** Help\
**Created:** [June 21, 2021, 10:42pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235 "2021-06-21T22:42:54Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![andrew-primer-e](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/andrew-primer-e/32/5139_2.png) [@andrew-primer-e](https://talk.observablehq.com/u/andrew-primer-e)\
**Post date:** [June 21, 2021, 10:42pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/1 "2021-06-21T22:42:54Z")

</div>

Can anyone recommend a simple and low-cost data hosting option that can be used for datasets that are too large to upload directly to Observable?

I have a ~500MB dataset that I’d like to analyse using an Observable notebook. In the past I would have stored this data in CSV format and analysed it using R Markdown notebooks, but I’d really like to use Observable to create an interactive web-based notebook instead. I’m a bit overwhelmed with all the different options available for hosting and would value any advice. The dataset is already in the public domain, so I am not concerned about keeping the data private.

---

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [June 22, 2021, 8:59am UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/2 "2021-06-22T08:59:31Z")

</div>

> **[Store Files with Storage](https://observablehq.com/@endpointservices/storage)**
>
> At last, upload files! First, provision a "bucket" to hold your files. Once logged in you can upload and download files using a storageClient. You login with an IndieWeb identity URL, which needs to be linked to an observableProfile to upload and...

or if you want a full on platform I think Firebase pairs well

> **[Ridiculously Rapid Software-as-a-Service (SaaS) development with Observable,...](https://observablehq.com/@tomlarkworthy/saas-tutorial)**
>
> sidenote: Unlike some APIs, Firebase API Keys are not secrets. (thanks @oleksandr\_now)

happy to realtime chat on [https://d3js.zulipchat.com/#narrow/stream/273726-observable](https://d3js.zulipchat.com/#narrow/stream/273726-observable)

---

<div class="post-metadata">

**Author:** ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)\
**Post date:** [June 22, 2021, 4:03pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/3 "2021-06-22T16:03:27Z")

</div>

Depending on your requirements you could also keep your file local by either selecting it via a file input (which simply allows the notebook to access the file, without uploading anything anywhere), or by running a local web server (e.g. via `npx http-server --cors`, if you have `npm` installed) and accessing the file via its local URL.

---

<div class="post-metadata">

**Author:** ![Cobus](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cobus/32/9670_2.png) [@Cobus](https://talk.observablehq.com/u/Cobus)\
**Post date:** [June 22, 2021, 4:08pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/4 "2021-06-22T16:08:53Z")

</div>

@mootari’s recommendation will definitely be the fastest, since you eliminate the network fetch. If you do need to access the file from the network then you should also look at Git LFS.

One thing to consider… your final visualization may not need all the data. Oftentimes, it is better (for the end user!) if the large dataset is reduced to something that supports the presentation of the data. You may need the large dataset to discover the story/patterns (which would work well with local files) and then you can reduce the dataset to a much smaller set that drives the visualization and have end users benefit from the speed of the smaller dataset.

---

<div class="post-metadata">

**Author:** ![andrew-primer-e](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/andrew-primer-e/32/5139_2.png) [@andrew-primer-e](https://talk.observablehq.com/u/andrew-primer-e)\
**Post date:** [June 22, 2021, 9:19pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/5 "2021-06-22T21:19:16Z")

</div>

Thanks for the suggestions!

I’ll use a local file for now and investigate Firebase once I’ve got the visualisations sorted.

---

<div class="post-metadata">

**Author:** ![Job](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/job/32/1185_2.png) [@Job](https://talk.observablehq.com/u/Job)\
**Post date:** [June 23, 2021, 12:51pm UTC](https://talk.observablehq.com/t/good-data-hosting-options/5235/6 "2021-06-23T12:51:23Z")

</div>

You probably already have stuff set up but for future reference, Mike Bostock has this notebook with a wrapper that makes it easy to open a local file and treat it as an Observable FileAttachment:

> **[LocalFile](https://observablehq.com/@mbostock/localfile)**

Might be useful for swapping out one for the other more easily without writing a lot of glue code
