🏠 back to Observable

why change recent URLs to use number instead of date?

Site Feedback
#1

This seems like a regression.

e.g. https://observablehq.com/recent?page=3 instead of
https://observablehq.com/recent?before=2019-05-20T16:39:40.365Z

Now the results are not stable over time, and those URLs are effectively meaningless after more than a few new notebooks get published.

Also, all of the old URLs are now broken.

3 Likes
#2

+1 <-- thinly veiled topic bump

#3

We made this change as part of the larger redesign in an attempt to improve navigating through notebook listings, so that you can:

  • See how many pages of results there are for a given listing or query.
  • Immediately jump to pages arbitrarily deep in the listing. e.g. If I am looking at all of my notebooks that mention “mercator”, and I want to see the earliest ones, I can now go directly to the last page.
  • Have friendlier URLs for listings.

With “random access within the result set” really being the most important quality here.

(It’s also in anticipation of us releasing improved advanced search capabilities, where jumping around within arbitrary lists of notebooks is going to become more important.)

Were you using bookmarks to specific pages of results with before= for something in particular? And if you were, can you describe the use case?

#4

All else equal, URLs that present consistent content are better than links where the content changes arbitrarily over time (making an exception for e.g. the front page of some list).

Some use cases:

(1) Start skimming through recent notebooks from sequential pages. Go away for a few days. Come back to the page and pick back up where you left off, instead of needing to scroll all the way back down from the top.

(2) See a few interesting notebooks that were published at about the same time. Email the link to your buddy, and have it still be valid a few hours later

(3) Enter some date in the past and see what kind of notebooks people were publishing on that day, instead of needing to guess the page number for that day.

Something not on Observable per se, but I find those infinite-scroll-with-no-URL sites incredibly annoying, and one of the big reasons is that there’s no kind of permanent URL to a point in the stream.

If I navigate to e.g. https://twitter.com/jashkenas and start scrolling down through the top 50 entries, and then I somehow navigate the browser away, there’s as far as I know no way to get back where I was without scrolling down from the top again. If instead the URL were updated to indicate the date of the tweet I was just looking at, I could go back to that URL at any point to pick up reading the stream.

If you have a page of search results for some complicated query where the order is not chronological and will change arbitrarily over time, then making a meaningful stable URL gets harder. In theory you could attach the date to the URL and as part of the search query filter out all newer results. But making those pages stable takes more thought and work, and might not be worth the engineering effort.

But for stuff that’s just “show me X page of stable results taken out of an append-only ordered list”, the URLs which keep track of the exact position in the list instead of just the position relative to the recent end are much more useful. YMMV.

3 Likes
#5

To add to the above: At first I was surprised by the date offsets in Observable’s pagers, but I’ve quickly come to appreciate them (this is coming from someone who implemented or integrated mostly counting-style pagers).
However, I understand that date based page offsets are pretty much incompatible with advanced (or, really, any) query filtering, so my two requests would be:

  • Have a date range filter so that I can restrict results to a time range (before, after, or both).
  • Salvage current date based offsets by offering perma links to result pages (i.e., static offset). I’d happily brainstorm practical solutions for this use case, as I can see applications for my day-to-day work as well.
2 Likes
#6

Just leaving some notes:

  • The term is “keyset pagination”.
  • Many articles suggest that this is “pagination done right” and offset based pagination should be avoided (I tend to agree).
  • In its pure implementation keyset pagination has no notion of page numbers.
  • Depending on the sort order a composite offset may be required.
  • A quick search suggests that page numbers can be reintroduced with the help of window functions (PostgreSQL >= 8.4, MySQL >= 8.0, MariaDB >= 10.2).

Edit: This could actually allow for a type of pagination that takes the sweet spot between page numbers and facets. E.g. if your sort order can be clustered into some meaningful represenation, this r. can be used to convey the distribution of the result set (mind you, these are drafts):

  • Sort by date (aka age):
    [Latest] [Feb '19] [Jan '19] [Newer] (May 6th 2018) [Older] [Oct 16] [Mar 16] [First]

[Latest] [1 week] [3 weeks] [Newer] (3 weeks) [Older] [13 weeks] [24 weeks] [First]
  • Sort by name:
    [First] [az] [gr] [Prev] (la) [Next] [ob] [ob] [Last]
  • Sort by relevance score:
    [Best] [7.3] [5.4] [Prev] (5.2) [Next] [2.1] [0.3] [Worst]
2 Likes
#7

This discussion of this general topic came up recently:

https://news.ycombinator.com/item?id=25547716