[rant]The current publishing/slug system is a PITA.

Feedback
1

Hey there,

<rant>
The entire notion of “persistence” that observable has is a huge half baked PITA without any real benefits.

  • Slugs are non reusable.
  • Unpublish is limited to 24h.
  • It’s hard and annoying to change notebook urls.

I’d be all in for this kind of “immutable reference” stuff if it was exactly that: immutable.
But notebooks are not immutable, they change a lot, and they can get edited and deleted willy nilly.

This does not solve the problem of code reuse and breakage, it only makes it more annoying to deal with, because it imposes restrictions which in no way alleviate the existing problems, and only create new ones.

Notebooks almost exclusively link to other notebooks latest versions, which is bad and breaks stuff. But it’s the way encouraged by the system, because versioned notebooks are not a first class citizen but an after-thought.
Visit any “@user/notebook@version” url and it will convert into the “d/whateverweirdstuff@version” slug.

The reason why this is so broken is because it complects “user convenient url” with “notebook reuse references”. Those should not be the same thing.
So here’s how it could be:

  • You can choose the notebook url freely, this is just for advertising it to other people (the @user/notebook stuff). Let’s call this the “public URL”.
  • Notebooks also have a “content URL”, which is based on the content of the notebook, just like a git hash. Notebooks are always accessible under that address, unless they are explicitly deleted.
  • When visiting a “public url” a redirect happens to the most recently published “content URL”, making the “content URL” the “real URL” of the notebook.
  • You can unpublish at any time, which will simply remove the “public URL”, but leave the “content URL” intact.
  • You can also delete the notebook, but you will get a modal reminding you that this might break other peoples code, and a list of other options you have that don’t break linkage, like unpublishing.
  • Whenever you import another notebook via a “content URL” the system just leaves it as is.
  • Whenever you import another notebook via a “public URL” the system automatically substitutes the cell with one containing the “content URL”.
  • You can add as much ceremony and tooling around the previous point as you want. “Hey we’ve noticed that your dependencies have updated! Do you want them to point to the latest version?”
  • Profit: No more breaking notebooks, no more annoying slug burning, no more “ah shit I don’t want this on my profile but I don’t want to break other code”, no more “I wish I hadn’t deleted that notebook I’d like to use that slug again”, no more “I really wanna cleanup all this fork mess but the slug is burned”.

Side note: In theory one could push this content addressability into referencing individual cells, but those “code as a graph” approaches, while totally awesome, and a super cool match for observables “code as a reactive graph” philosophy, somehow always fail. Maybe because they are too fine grained for the human mind, with whole “file” and “notebook” versioning being easier to reason about .

</rant>

Rock on!
JP

4 Likes
2

Thanks for the feedback, and sorry for the frustration.

Here are some things we’re working on to make this better.

First, proper version pinning for imports (and requires). This will allow notebooks to capture exact versions of imported notebooks and their transitive dependencies, similar to how yarn captures exact versions in a yarn.lock file. This ensures that notebooks keep working into the future: imported code will never change. Internally it will work similar to how you describe: imported notebooks are referenced by id@version (e.g., 09403b146bada149@257 for the current version of @d3/bar-chart).

Second, editable URLs. We’d like to show you what the URL will be before you publish, and give you an opportunity to change it if necessary. And internally we already support changing the URL of a notebook with redirects (e.g., @mbostock/d3-bar-chart redirects to @d3/bar-chart), so we’ll allow you to change the URL after publish, too. (If you want to change the URL of a published notebook in the meantime, please email support. We’d be happy to help!) We’re also thinking about allowing human-readable URLs for private notebooks and for shared notebooks that aren’t listed publicly or discoverable via search.

That said, we’re not planning on letting you re-assign a URL (slug) to a different notebook. Even with version pinning, I think it would be confusing to have a different association between the human-readable name of a notebook and its identity/history (e.g., @​d3/bar-chart@256 meaning something completely different). The human-readable name matters too, and it’s easier to reason about a sequential history than large hexadecimal identifiers (or content hashes). But as an author you can always completely overwrite the content of a notebook as a new version, or trash the old notebook and publish a new notebook at a different URL (slug).

2 Likes
3

Glad to hear. :slight_smile:

Some feedback on fixing the slug:
The issue with that is that it doesn’t prevent the confusion, it just makes it very hard to work with. Right now there is no way to remove a fork from a notebook, having the ability to create a “fresh” notebook and associate it with a URL would be a fix for that.
Also porting over one notebook to another, cell by cell, only makes this a huge PITA, without actually removing the confusing part of “everything changed”.

I get that observable is meant to be used by “not as technical” people, but these “fixes” make it unnecessarily hard for power users to publish clean work, without confusing fork histories.
This also doesn’t address on how to get burned slugs back once you’ve deleted a notebook.

My point is that this will result in a pile of “workarounds” and “hot-fixes” to solve the symptoms, because it doesn’t address the root issue: A notebooks primary identifier should not be the name given to it on publication.

Git solves this extremely well, and I’d wager that everybody who uses observable is also capable of using git.
I recon that it would be a lot easier to explain somebody how observable manages changes, merges and forks by telling them “it works just like git”, than to explain and learn a completely different system, that kinda sorta works like git, but not really.

Addendum:
Sequential version identifiers are a lie. One that gets bigger with the number of undos, merges and forks you do. Fork/Merge/Change graphs are hard enough as is, pretending they don’t exist and hiding them behind a sequential ID, might look easier, but in reality makes it harder to work with them.

Addendum2:
The pinnacle of usability would be to be able to ‘git clone’ notebooks. Throw in an ‘observable’ command line script that allows you to locally launch notebooks with the editor, and you’ve turned observablehq into the github of notebooks. I know that you’re probably weary of tearing down the walls to the garden. But in my experience tools like Observable profit massively from network effects and compatibility.
If you make it the easiest and most convenient to use people will flock to observablehq to publish their notebooks, if you nail it down, they’ll go somewhere else.
With the runtime already open source that genie is already out of the bottle imho.

I’d love to have all of my libraries in any programming language be accompanied with a notebook that hosts living documentation.

Hell I’m already writing all of the GUIs for my projects in Observable, might as well ship the notebooks with the projects.

1 Like