# Error monitoring for notebooks!

**URL:** <https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757>\
**Category:** Show and tell\
**Created:** [October 29, 2021, 7:30pm UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757 "2021-10-29T19:30:54Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [October 29, 2021, 7:30pm UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/1 "2021-10-29T19:30:54Z")

</div>

I think I have a good solution for keeping notebooks running. With a few tweaks [sentry.io](http://sentry.io) works in userspace, and it will collect and summarize where errors are occuring, and will include what OS/Browser/Device etc things are breaking on, and what was happening leading up to an error. It’s a very professional solution to the problem, and provides a single place to monitor all your notebooks! I put some screen shots in the following notebook

> **[Observablehq.com Notebook Monitoring with sentry.io](https://observablehq.com/@endpointservices/sentry-io)**
>
> Sick of broken notebooks? Don't be that guy! With some minor configuration you can have sentry.io monitor notebooks for unexpected errors, so you get informed fast when something breaks, and get to the bottom of those tricky to repro issues. Example...

Best thing? For low traffic single developer use cases… its FREE

---

<div class="post-metadata">

**Author:** ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)\
**Post date:** [October 29, 2021, 9:23pm UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/2 "2021-10-29T21:23:15Z")

</div>

Caveat: Only errors that are not handled by Observable’s Runtime can be logged.

---

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [October 30, 2021, 8:02am UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/3 "2021-10-30T08:02:24Z")

</div>

I am digging into this more. So while `{throw new Error()}` is not reported `{eval("foo")}` is. Sentry does report even handled errors IF it occurs in some well known place like eval or requestAnimationFrame.

So yeah, some cases are missed, but a lot of cases, even in userspace code, are detected. Errors in promises or generators being a big category of common userspace errors.

It’s a great point though, it’s not 100% coverage, so I added a limitations section. Still, if you are looking for an easy way to catch 80% of unexpected failures this is a good way IMHO. Its a vast improvement over nothing and it only takes a few minutes to setup. It also picks up errors on devices you might not own or browsers you might not have access to, so its a cheap way to do cross platform testing, even with its limitations.

---

<div class="post-metadata">

**Author:** ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)\
**Post date:** [October 30, 2021, 10:11am UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/4 "2021-10-30T10:11:26Z")

</div>

> [@tomlarkworthy](#):
>
> `{eval("foo")}` is

Are you certain? I don’t see how Sentry would be able to catch that. Be aware that while testing I triggered some errors from the JS console, outside of Observable’s Runtime.

> [@tomlarkworthy](#):
>
> an easy way to catch 80% of unexpected failures

That number seems far too generous. You may be able to detect unhandled rejections for internal fetch calls (i.e., where the result is processed inside the same cell that made the request), but any promise rejection that is returned directly to the Runtime should be unloggable. That includes failing imports, requires, and top-level fetches.

---

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [October 30, 2021, 11:20am UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/5 "2021-10-30T11:20:01Z")

</div>

OK yeah, I have done more testing and I must have got my wired crossed with your tests last night. eval("/") does not get caught if its in a cell. Nor do require(…) errors.

You can manually wrap and report, thought that super sucks (especially as you might need to add in some awaits) so I wonder if there is another way

so this reports but I don’t fancy destroying all my notebooks to do that

```auto
{
  try {
    return await require("gibberish module that is caught");
  } catch (err) {
    Sentry.captureException(err);
    Sentry.flush()
    throw err;
  }
}

```

---

<div class="post-metadata">

**Author:** ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)\
**Post date:** [October 30, 2021, 11:42am UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/6 "2021-10-30T11:42:16Z")

</div>

_Edit: I missed the code in your reply, which is basically the same as below. So my recommendation would be to simply wrap the logic in a helper. So anything critical would be called as, e.g., `sentry(doStuff)`._

Assuming that you primarily want to detect broken external resources, you could do something like:

```auto
async function logErrors(p) {
  try {
    return await p;
  }
  catch(err) {
    setTimeout(() => { throw err }, 0);
    throw err;
  }
}

```

and then (invalid package name for testing):

```auto
d3 = logErrors(require('d3-foo'))

```

```auto
d3 = logErrors(import('d3-foo'))

```

Instead of the setTimeout you could also log straight to sentry.

---

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [November 2, 2021, 8:43am UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/7 "2021-11-02T08:43:33Z")

</div>

Yeah, I don’t like changing source code in order to satisfy monitoring. Adding an independant cell to add monitoring is ok, but having to change every unrelated cell in the notebook is too much.

So there are two major gaps in monitoring. 1. anything processed by the runtime, and 2. anything that errors out before the the SDK is loaded.

So here is a solution I will try out next that hopefully solves both gaps.

> **[Notebook Health Check](https://observablehq.com/@endpointservices/healthcheck)**
>
> This notebook executes other notebooks through an embedded runtime and looks for errors thrown by cells. If an initialized sentry.io SDK is found, it is used report those errors. You can the error scan manually, or through a HTTP endpoint. The...

This notebooks runs another notebook through a private runtime, so it can see all the errors. It buffers them and will regurgitate them if it finds a configured sentry runtime. It also exposes a http endpoint to run the notebook, so all you need to do is plug that endpoint in an cron-like availability monitor. I am using UptimeRobot

Better?

---

<div class="post-metadata">

**Author:** ![tomlarkworthy](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tomlarkworthy/32/5940_2.png) [@tomlarkworthy](https://talk.observablehq.com/u/tomlarkworthy)\
**Post date:** [March 31, 2022, 3:21pm UTC](https://talk.observablehq.com/t/error-monitoring-for-notebooks/5757/8 "2022-03-31T15:21:47Z")

</div>

I had a problem in production which would have been solved much quicker had Sentry been able to detect exceptions caught by the runtime.

So I have developed (on top of @mootari’s runtime sniffer) a way of cell errors with a catchAll handler [Detect notebook runtime errors with catchAll((cellName, reason) =\> {...}) / Tom Larkworthy / Observable](https://observablehq.com/@tomlarkworthy/catch-all)

The sentry notebook has been upgraded now to also report cell errors! Thus closing a gap in the monitoring coverage.