Is it possible to encapsulate Secret-based code for public notebook use?

Community Help
1

I currently have one public notebook that uses static data that I want to convert to grab info from an API. The major issue is that the API has the key embedded in the URL and I am trying to hide that form others. In a private notebook, I can retrieve the data as a xml_string that I can convert to JSON and work with.

I was thinking that I could hide the URL with an import { MOHW_DATA } from <id of private notebook> but that seems impossible. I can’t use Secret’s because that is for private notebooks only, and besides just posting my key in a public notebook, I don’t understand how I can overcome this.

MOHW_Data = (start = date_start, end = date_end) => {
  const base_url =
    'http://openapi.data.go.kr/openapi/service/rest/Covid19/getCovid19SidoInfStateJson';
  const cors_url = 'https://ciscorucinski-cors.herokuapp.com/';

  const key = Secret('KEY_DATA.GO.KR');
  const date1 = format_url(start);
  const date2 = format_url(end);

  const xml_url = `${base_url}?serviceKey=${key}&startCreateDt=${date1}&endCreateDt=${date2}`;

  const xml = fetch(`${cors_url}${xml_url}`).then(res => res.text());

  return xml;
  //   const json = xmlToJSON(xml);    // TODO: conversion of XML_STRING to JSON always fails in this function. It works outside.
  //   return json.response.body.items.item;
}

My goal is to hide the url which has my API key embedded in it, but allow the data to be retrieved from a date range. To retrieve your own key, you have to apply through a korean website, there is no “demo” key, and I want the public notebook to have the data displayed, so am not sure if a placeholder is good enough.

I am open to a new approach if my current approach is infeasible.

I was looking at the following. Not sure if this will work yet. A lot of it goes over my head right now due to the complexity. Achievement Unlocked: Secrets in public notebooks

1 Like
2

Yeah there are two options:

  1. private key and authorised notebook (e.g. enter a password to get the secret key)
  2. private key, public notebook (anybody can use the notebook but you want the key to stay secret)

I think you want 2 right? 1, can be achieved by encrypting the key and an authorised user supplies the password to decrypt it. The nice thing with 1. as this can be done in the open without additional infrastructure.

For 2 we need a server to handle the secret, so it never enters the browser runtime. So you can do 2. with a combination of serverless-cells and secret-manager.

Use secret manager to save a secret. Then inject the secret into a serverless cell runtime. Checkout “example_injecting_secrets” in serverless cells

You serverless-cell handler would call the API with the key, and return the results. You main notebook would be calling your serverless-cell URL without a key

1 Like
3

I hacked up an example though I cannot test it without the real API key. Nor do I have the correct formatDate implementation.

3 Likes
4

Thank you for showing this and the different options! This is such an interesting way of solving this. I will take a look at this soon and see what I can do and if I have some more questions.

5

The other thing to remember is that you create a public API to their private API when you do this kind of thing. So to reduce the usefulness of your API to other actors you should reduce its functionality to the minimum required to serve your public usecase.

Which I think you kind of have already as you can only query it by dates now. I would put limits on the date range too, so no-one can force a a fetch of an absurd amount of data in one go. (think 1970 to the year 3200)

1 Like