We are a startup and getting a couple dashboards going in Observable Framework + Observable Cloud. So far so good. Our data lives in AWS Athena and Google BigQuery and we have data loaders working well. We are a pretty basic when it comes to data pipeline. TLDR get it into S3 or BigQuery. Playing with dbt but… probably unnecessary in our use-cases.
I share all that to say, in reality, the cached data in Observable’s generated Parquet files are effectively the data source I want to drive BI and alerting automation from.
Does anyone have examples of loading these Framework generated data files to other automation? For example: Slack notification of metrics going out of spec or email summary of a dashboard?
Curious to brainstorm with others!
I think the thing you’ll need on the Framework side is the dynamicPaths option, which would allow you to place the files in deterministic locations. From there you’ll need a secure way to access the files without making them public, which is what the signed URLs feature of Observable Cloud is for.
I’m curious to hear more about what you build with this.
Thanks @mythmon for the lead on dynamicPaths option, I missed that before. I was gonna scrape the page source for registerFile and registerTable calls 
We are already using signed URLs so should be good there.
The specific use-case I am thinking about right now is replacing some automated emails we have about reporting AWS costs (scheduled lambda calling some AWS APIs) with this. That way, I can centralize and standardize the data loading from external systems from within our data app, and query it in a more straight forward way for other automation.
I guess traditionally, this would be handled through different BI ingestion infra into a data warehouse and transform it for reporting via dbt or similar. I guess the main downside to my proposal here is that I am creating an external dependency to the data loader format of data…
I am happy to show & tell if others find it interesting!