I recently hit a token limit working with the AI Assist and a new.observablehq notebook. My personal account setting in new Observable did not have a Billing tab and adding a new API key to my old Observable did not make any difference. (Same error of limit reached.) Is there a workaround or will this be soon resolved?
Hi John! For 10x higher AI token limits in the new Observable, you should be able to upgrade from the old Observable workspace settings screen:
For you that should be here: https://observablehq.com/team/@shadoof/settings
(For others’ reference, you can go to Sign in | Observable)
If you didn’t see that option before, you might’ve been looking at your personal user account settings at https://observablehq.com/settings, instead of your personal workspace settings. (The user is about identity; the workspace holds your work.) We started making that distinction a couple years ago and I think it has been very confusing; we are re-unifying the settings in the new Observable. But for now, the billing is still on the old site, so it still has that split.
Thanks @tophtucker . I did upgrade and also finished what I was working on but have managed now to hit the new limit. Now, I am still a bit confused because, as I understand it, you also allow a ‘Bring Your Own Key’ set up, which I have duly set up. Nonetheless, I am still getting the limit reached error. I have not yet refreshed the page of the notebook I’m working on (not wanting to lose the exisiting chat). Would doing this make the Bring Your Own Key work? Or am I missing something?
Hi @shadoof. Thanks for upgrading to Pro! 
We appreciate your business.
Apologies for the inconvenience caused by hitting our rate limits. It does appear that you’ve reached the daily token limit for Pro, too. Limits are computed using a rolling 24-hour window, so you should get more usage again in about 20 hours.
Unfortunately we only offer a “Bring your own key” (BYOK) plan on Observable Desktop, not on web. And this was intended only as a temporary measure during the Technology Preview phase so we could rollout the Agent sooner before we had the necessary infrastructure on web. Now that we’ve go the Agent running on the web, we plan to phase out BYOK on Desktop eventually. That said, we do plan to support BYOK for Enterprise customers (and support for local models etc.). Let us know if you’d be interested.
The current limits we have in place are provisional. I imagine we will revise them as we better understand usage. Limits are unfortunately necessary due to the cost of AI.
We also need to get better at optimizing token usage by agent. Most urgently, we need to implement conversation summarizations. If you have a long conversation (either in a chat or in a notebook), the context window gets pretty big and the token consumption can grow quadratically, although that’s mitigated somewhat by caching tokens.
Therefore one way to significantly reduce your consumption now is to favor smaller chats. You can do this by creating new chats or by resetting (reloading) the agent panel in notebooks. If you stick to smaller chats, you should be able to get more replies with fewer tokens (and it’s better for the environment, to boot).