Republishing code on observable

Community Help
1

I’ve been slowly pulling into observable a few functions and tools I use very frequently that aren’t packaged up in a manner that makes them easily importable to observable. I copied in mattdesl/eases a couple days ago. I thought I’d seek feedback on whether this is a Good or Bad thing or whether I could be doing things differently/better. Here are a few thoughts:

Pros:

  • When imported into a notebook, import { cubicOut } from '@rreusser/eases' links directly to the implementation, which I really like compared to tracking it down via npm then github: eases / Ricky Reusser / Observable (subtle feedback: the page scrolls to the anchor tag before the notebook executes, which is not a meaningful location after figures have rendered. I suppose the page may never “finish”, but it often ends up in the wrong place after much less than a second.)
  • can add additional documentation like plots that help communicate the meaning without the plotting library getting pulled in as a dependency of dependents. (yay!)
  • no need for require tricks, e.g. Module require debugger / Observable / Observable
  • doesn’t depend on third party services like bundle.run (I like to think things will be around for a while, but all of my codepens from the wzrd.in era are now broken)

Cons:

  • Duplicates code that’s already published elsewhere instead of working back upstream the build steps that would be required to use them via unpkg or similar.
  • Puts things other people created under my name, which, even though I’ve tried to be very clear about attribution and licensing, feels, I don’t know, muddy?
  • I’m still 10% fuzzy on the resulting license. By importing code and presenting the original license, have I created a new work that carries with it its own license?
  • Honestly just feels kinda redundant/gross. :man_shrugging:

Just curious what thoughts or opinions people might have!

3 Likes
2

Personally, I favor publishing packages to npm when I’m deliberately trying to make something reusable, and I favor notebooks for ad hoc experiments, learning, examples and documentation. Of course there’s a blurred line between the two, and things often start out as notebooks that later become libraries.

On the specific matter of eases, d3-ease will work out of the box if you want something to require:

d3 = require("d3-ease@1")

And of course you can find some examples and documentation here:

https://observablehq.com/collection/@d3/d3-ease

2 Likes
3

For what it’s worth, I find one-off functions implemented in notebooks a lot easier to comprehend than random bits of D3 or similar libraries.

First there’s the reduced friction of clicking on the import statement to go directly to the implementation. With the library version I usually have to go do a web search for the name, then click through a few github pages, then try to navigate around in the project to find what I’m looking for.

Beyond that, D3 sometimes does tricky stuff that isn’t directly shown in the file where a specific function is implemented (this is a bigger problem in Javascript than in some other modern languages because there is usually an additional compilation step inserted between the source files and the target library but there’s no one standard for how it should work, because Javascript hasn’t really standardized a way to handle imports or namespaces). So in practice instead of searching in one webpage I have to download all the code for the project and spend more time reading code, grepping through for random function names, and trying to figure out how the control flow works, what the internal abstractions are, etc.

I’m not trying to bash on D3 – for the context of a large library packed with stuff it makes sense to build abstractions that reduce boilerplate, provide for richer APIs, etc. – and this is by no means unique to D3. I was trying to figure out how some of the functions in https://github.com/stdlib-js/stdlib worked, and there’s a whole lot of time/brain overhead involved in digging down to understand the part I actually care about.

I guess what I’m trying to say is, the trade-off is pretty different between wanting to understand just a single function vs. understand the philosophy and design of a big library. For just a single function, being hosted in a notebook is pretty convenient. Of course there are also significant downsides vs. being a proper (e.g. github) project, like the lack of an issue tracker, less refined tools for navigating history, lack of clear releases, relative inaccessibility from outside the context of notebooks, etc.

2 Likes
4

One thing you might want to do is add links directly to the relevant lines at Github for each of your copy/pasted functions.

1 Like
5

Thanks for the feedback, @mbostock and @jrus!

Zeroth of all, I want to make it clear, as @jrus did, that if I critique any projects or patterns, it’s only with immense gratitude and the hope that by understanding them better and putting myself in a position to contribute, I can pay back some open source debt and help improve or help others use the projects that have made it even possible for me to do the work I do. :bowing_man:

First of all then, thanks for pointing me to d3-ease, @mbostock! Looks wonderful and a bit more feature-complete than mattdesl/eases. I’m only just learning d3, but the more I learn, the more I like the patterns and organization of the modules. I truly appreciate how much went into its creation and documentation! That being said, I find that import {...} from '...' feels very natural within observable and, whether intended or not, makes possible an immediate connection to documentation much like you’d find in a product like MATLAB. Window functions are similar random thing I think would lend themselves well to this pattern. Or morphological operations. I think all of these things have in common that they benefit from readily accessible visual documentation being a core part of the development process. More sophisticated modules or modules that pull in other dependencies are more complicated and probably don’t fit into this pattern.

I think maybe part of the friction I’m fighting is that there are quite a few camps on what good JS development looks like and I, someone who uses and contributes a bit but doesn’t “own” any project, have always felt a bit caught in the middle since people use JS to solve every problem in every environment so that it doesn’t have a de facto foundation like numpy/scipy or even Eigen. To write modules that interoperate with others means lots of little battles like whether to create micromodules (stackgl?) or mesomodules (d3?; clarification: utterly made-up term because it’s not technically a monorepo, I think?, but shares nearly the same philosophy, which IMO is a great one) or monorepos (stdlib?), whether to favor require or import, build processes + dist versions or just code. Whether to deal in typed arrays as the lowest common denominator or a higher level construct like ndarrays. If I just contribute PRs to add UMD dist bundles, it’s challenging to get that past PR review and could very well lead to ten copies of ndarray getting pulled in if not thought through carefully. Perhaps import solves this, but @kgryte has rightfully pointed out that backward compatibility is not to be taken lightly.

Observable is very interesting to me because it’s a unique and novel platform on which a lot of these things can be built in a way that feels completely natural. I think it’s the first time more powerful uses of JS that put communication and portability at the forefront have even felt viable to me. And I emphasize platform because it’s wonderfully agnostic about which libraries you use on top of it—which has led to some really neat uses.

Perhaps the challenge I’m trying to voice then is that I’m trying to understand how to build out—or just expose—high quality pieces that fit into this world in a way that utilizes what Observable has to offer. I think It’s pretty common knowledge that I find stdlib a viable candidate for a project that addresses many of the truly difficult pieces of building a coherent environment head-on. Any sort of endorsement or preferential treatment of stdlib by Observable makes no sense, but I wonder if exposing such tools in a coherent manner is something that can be accomplished in userland. For simple modules, it’s possible to copy/paste code and produce something that’s nice and, at the very least, not a maintainability disaster. For larger modules, that’s not the case.

Anyway, nothing to stress about, but I just wanted to voice the sort of things that roll around my head and cause me to lose a bit of sleep from time to time, but which I don’t generally act on because I simply haven’t figured out the answers yet.

Thanks again for the feedback and for producing an environment that gets my mind turning on things like these!

5 Likes