What type of values are you planning to check? Scalars (number, string, bool …), or also objects like Array, Map, Object …? If the latter, how would you perform these checks (e.g., by reference, or by checking specific key/values)?
Is the purpose of the check to guide your students while they work on a notebook, or to verify their work after they’re done? Would it be acceptable if only the link-shared/published version of their notebook could be checked?
Variable Types: I was planning on checking numbers, strings, booleans, as well as arrays and possibly objects. I’m not looking for a solution to every possible case, but started working on it here https://observablehq.com/@uw-info474/utilities#check_answer Purpose: My first/primary use case is for students to get feedback while they work on a notebook, but it would be interesting to also think about applying it to checking the notebook (I’ve done some work with Jest testing for assignments in other web development courses). We’re working in an Observable team, so most notebooks remain private but shared within the team (ideally it would work without publishing it).
I couldn’t come up with a better solution, and frankly I’d prefer Mike’s solution over something fragile that can break at any moment (and in turn frustrate students).
But perhaps we can take a step back and focus on the problem that you’re trying to solve:
Why do you feel that the possibilities within Observable aren’t sufficient?
Is there some student feedback that you can share? E.g., are the error messages too aggressive/offputting, or too vague?
Is there a specific scenario that we could use as basis for this discussion?
Thanks so much for your time and thoughts! To step back, I’ve realized through multiple quarters of teaching using Observable that data wrangling is a big pain point – in turn, I wanted to develop structured notebooks for guided practice. As a bonus, I wanted students to know if they got a “correct answer”.
In my current solution, my only real pain-point (as an instructor), is having to to define each cell as undefined to provide a “cleaner” (i.e., error free) set of feedback to students as they move through the notebook – see here, where I have to pre-define the cell:
// What is the mean number of attendees? `mean_attendees`
mean_attendees = undefined // this is the line I didn't think I'd have to write
Obviously not a big pain point, but it did surface an issue that I found curious – that I could use eval() for JavaScript variables in way that didn’t work for Observable cells (as described here).
I appreciate better understanding the runtime and approaches you all have described. I think the smoothest way forwards (as an instructor) that gives students the clearest experience in each notebook is to simply define cells as undefined (I find it a bit easier than Mike B’s solution to repeat update("var_name", var_name, invalidation) for each variable.
Thanks for your reply! I hadn’t tired of the conversation, but signed off for the holiday break. In terms of the “pain point” experienced, I’d say it was more like #1 (students should create the cell and name the variable themselves). But, because I have to tell them the name of the variable, it doesn’t actually create much more work on my end.
FWIW, I really like the idea of using the REPLACE_ME variable! Again, thanks for all of your thoughts, they were helpful and interesting!
While the eval hack mentioned in February was mitigated fairly quickly, the original monkey patch hack still works.
Today I rewrote and vastly simplified Accessing a Notebook's Runtime / Fabian Iwand / Observable to handle cases where the Runtime would not get captured when importing into another notebook. The implementation now triggers a recomputation explicitely instead of relying on some implicit behavior.