Suggestions to streamline the process of reporting site issues and bugs

Feedback
1

So I’m an Interaction Designer, It’s a sunny autumn day, I have some spare time, I had a ton of fun playing with an Observable notebook yesterday, and feel like contributing in a way that fits my knowledge domain :slight_smile:. I just submitted a bug report, and while the experience was ok, I immediately saw some spots that could be polished to make the process easier. The lower the threshold to writing a useful bug report, the more likely people are to report them and the quicker they are found and fixed, no?

So here a case study of submitting a bug report, based on my own experience. It breaks down and (over)analyzes it to see where things could be streamlined and what some possibilities solutions there might be. Hope it will be of use!

First, let’s agree that:

  • the stakeholders in this scenario are the end-users and the developers (people may have both roles of course)
  • for both, the desire is that bugs should be discovered and fixed easily, costing as little energy and time as possible, without the experience causing too much negative emotions.

To break that down in relation to reporting bugs, in no particular order:

  • it should be easy for the user to find out where to submit a bug
  • it should be easy for the user to see if a bug was already submitted/fixed
  • users should feel comfortable to share perceived issues on the one hand, without encouraging entitlement on the other
  • the reported issues should be expressed in a way that makes the problem clear to developers
  • the reported issues should be expressed in a way that does not make the developers defensive
  • the flow of the conversation between developers and users should be a pleasant process of trying to find a solution to the problem together

These may sound obvious, but writing it out ensures we’re on the same page. They’re not all equally important of course, and it’s not up to me to decide the importance of each.

Now to exhaustively descibe the steps I took to submit a bug, how I experienced it and what my thoughts/expectations were along the way:

  1. I noticed that the keyboard shortcuts for moving a cell up or down in a notebook (ALT-CMD-UP/ALT-CMD-DOWN) are not working on my Linux desktop.
  2. I wished to report this issue in the appropriate place, and looked for a link that let me report this bug
  3. After scrolling down I saw a number of candidates:
    • Help - not clear if this links to documentation or something else. Turns out to link to these forums
    • Open Source - does this link to the source? No, it just lists the open source technologies used
    • a Twitter icon - used only for promotion, or also for user feedback?
    • a GitHub icon - “traditionally” a place to report, bugs, right?
  4. I first tried the GitHub icon. It links to the obervablehq account on GitHub, presumably because it mainly has the function of just linking to the source code. It shows a number of repositories.
  5. The specific purposes of these repositories and relations to each other were not immediately clear to me. The names gave some hints, but not enough to be confident of where to go next.
  6. As a result, it was not immediately obvious for me where to open an issue for this particular bug. I had to open various repositories to get some impression of what they are for. This took time and energy - I was not trying to figure out how everything is glued together, I just wanted to report a keyboard shortcut isn’t working as intended.
  7. I still put in the effort to read through the repositories, because in the back of my mind I was worried about wasting people’s time and getting a dismissive reaction of opening a duplicate or going to the wrong repo (alternatively, some users may not care and just submit in the first repository they see, which probably isn’t what developers want).
  8. I remember past experiences with other grumpy programmers who are not very receptive to issues being opened for various (often understandable) reasons, so I hope that if I’m in the wrong spot I’ll be gently pointed in the right direction, and that I’m at least appreciated for trying to help.
  9. After spending some time cautiously looking around I settled on opening an issue in the notable-runtime repository, since it is a bug in the front-end interface and that seems to be the place with code related to that.
  10. Later, when I wanted to learn more about the notebook API, I clicked Help, which linked me to this place.
  11. After signing up and finding what I was looking for, I noticed the forum category Site Feedback. While fitting, it does not explicitly mention that one can report issues through it.
  12. Out of curiosity, I then searched for keyboard shortcuts and found a few discussions, including an earlier bug-report with a healthy discussion.
  13. This made me wonder about the other options I did not try.
    • clicking open source takes me to a page with just a list of used open source technologies
    • opening the Twitter account does not make it clear to me whether mentioning site issues through that channel is appreciated or not.

Analysis and thoughts

Giving clear guidance would give more confidence to users that they are doing things right, making it more likely to go through the hassle of reporting an issue in the first place. Among other things, guidance can take on the form of:

  • “findability”, with clear indications where to go,
  • what the developers need for a useful bug report,
  • friendly reminders that someone might already have reported the issue, and advice on how to check for that
  • a reminder that developers are people too and to be nice
  • a thank you for taking the time to report an issue

Having multiple possibly correct channels for feedback (steps 2+3) creates confusion and requires more user effort to get through. If they are all acceptable channels for feedback, then this is currently not communicated to the user.

Having a central location intended for reporting issues would not have to mean the other channels are not acceptable ways to give feedback; one could simply point to it from all of these locations. It also makes it possible to link from other locations (like the Open Source page)

Having a central location also makes it easier to find out whether someone else already encountered and/or found a fix for this problem. That should reduce duplicate issues.

The Site Feedback category on this discourse forum suggests that this place is already intended to be a first location to submit bugs. If so, it is currently not immediately obvious when looking for a place to submit feedback from the notebooks.

Some GitHub projects create an issues-only repo, from which issues can be propagated to the right sub-repo. For example, one could be created called observables-issues, with the description “a meta-repository just for reporting bugs encountered while using Observables notebooks”. A benefit would be that issues across repositories and reference each other easily. Linking to forum topics from GitHub and back is more cumbersome. The downside is that it requires a GitHub account, which increases barrier to entry for people who don’t already have one.

So with all of that, here are some suggestions:

  • Make it more clear and easy to find where to report issues and where to give feedback
  • Decide on a central location for reporting issues. Some options:
    • The “Site Feedback” category on this forum
    • An issues-only repository on GitHub
    • Both, kind of: the forums as a place for “regular” users, GitHub for people already on GitHub and for the developers to keep track issues raised on these forums and elsewhere.
  • Make it easy to find to whatever central location is chosen by linking to it directly from:
    • the Observable notebook interface, with:
      • the ? help pop-up: “Read the introduction, ask a question or report a bug.”
      • a directly clickable link. Alternatively: a directly clickable link that opens the ? pop-up, which contains the link to report bugs (for those who clicked away the instructions and cannot find the shortcuts again
    • these forums, when an off-topic issue is brought up in an conversation, with the suggestion to search for similar issues and opening one if it cannot be found)
    • Twitter conversations when users report issues.
    • the observableshq GitHub user, by mentioning it in the profile bio (there currently is no profile bio, by the way - might be nice for people who stuble upon the GitHub user without further context).
    • the “Open Source” page, by extending the opening paragraph to link to both the observableshq GitHub user and whatever becomes the central “issues” channel like so:

      “Observable wouldn’t be possible without the code written and freely shared by the open source community. This web application includes code from the following modules. (If you came looking for the source code, click here. If you were trying to report an issue, click here)”

  • Create a template for the user when opening a new thread/issue (this is possible on GitHub, I presume Discourse also allows for this). The wording should be welcoming and friendly, and could contain helpful suggestions, like:
    • how to check if it is a duplicate issue,
    • how to structure their message to make it most likely others can help them,
    • if it is a bug, which steps are required to reproduce it bug

Ooof, ok so that turned out way longer than expected! If you read all the way to here, thank you for you attention. I hope this is useful feedback!

Thanks for making Observable. I hope it keeps growing and improving!

/Job

edit: typos

2

The desired place to report issues is on this forum, or if needed, privately by emailing us at support@observablehq.com.

GitHub issues are specifically for our open-source libraries, such as our runtime. Things like keyboard shortcuts are issues with the user interface, not a specific library.

We’ll try to make that clearer. Thanks for the feedback.

1 Like
3

Thank you for clarifying!

Does that mean I should open the mentioned issue here again? Or perhaps find an appropriate existing topic about keyboard shortcuts?

4

Message received. No need to refile it, but if you have a future issue, you can start a new thread here.

5

I’ve pinned the Welcome to the Observable Forum topic globally. This should be the first thing everyone sees when they enter the forum, and I’ve edited it to describe how to report an issue, which is to open a topic in the Site Feedback category.

Per your other feedback, we’ve also changed the keyboard shortcuts for moving cells on non-macOS platforms to Control-Shift-Up and Control-Shift-Down. (They’re still Option-Command-Up ⌥⌘↑ and Option-Command-Down ⌥⌘↓ on macOS.) And we’ve moved the “Help” action to the notebook menu (⋯) from the user menu (avatar).

1 Like
6

Nice! Thanks for picking this up so quickly!

I know this is nitpicking, but may I suggest using sub-headers to make it even easier to spot the “sub-topics”? It again helps discoverability, and also makes it immediately clear to new visitors that yes, this is the right place to report issues. Something like this:

1 Like