I want to learn D3. I don`t want to learn Observable. Is that ok?

And to play around with an example directly in the browser, check out blockbuilder.org. Simply append the gist path to the domain name, e.g.:

https://blockbuilder.org/mbostock/1b64ec067fcfc51e7471d944f51f1611

Please be aware though that many of the examples use old versions of D3, often with major API differences.

1 Like

Like others, I came to this thread hoping to find a basic HOWTO for running d3 code inside a simple web page.

Well, youā€™re wrong. It is a barrier to collaboration, because I wonā€™t start until I know this thing is useful. Iā€™m not convinced it is.

The whole point of having a javascript engine is so that we can use data from our web applications. Iā€™m not interested in a dynamic view of static data, Iā€™m interested in a dynamic view of dynamic data.

Before I do 10 minutes of work learning your new Matlab/Maple/Mathematica/Juypiter workbook engine, which Iā€™m sure is very nice, I want to know if I can make a basic plot inside my web application, and how much work it will take to get there.

I donā€™t want to learn your software. I want to know if your software is any use to me. The official answer seems to be ā€œNoā€.

Either give us a basic ā€˜getting startedā€™ demo in pure HTML/Javascript, or stop calling yourself a library. Every response makes clearer that d3 is not a library, itā€™s a platform.

2 Likes

I guess itā€™ll take four lines of code, plus an import:

After that, just embed the code to get it into your application.

Why so easy? Because I imported functionality from this awesome Plotter notebook - collaboration at work! Need to incorporate that functionality into more advanced work? No problem. Need to use D3 for more complete control and flexibility? Again no problem. Hereā€™s one of those embedded into a webpage.

For the record, I do understand the frustration as my first exposure to Observable was while trying to wrap my head around some D3 code I found in a Google search. Even though I had been programming with D3 for several years, it was hard to figure out the code since it seemed intertwined with new Observable functionality that was then unfamiliar to me.

If thereā€™s one mistake made by the Observable team that this thread has uncovered itā€™s tying the D3 documentation too tightly to Observable. I guess that @Fil said as much in his response.

To be clear D3 and Observable are two totally different things. I think itā€™s fair to characterize Observable as a platform. D3 is certainly a library - one of many that the Observable platform makes it easy to use.

4 Likes

Hi Nathaniel,
In case you want to see a very simple example of how to use D3 code in a browser, here is a quick example. (I copied the code from this notebook, in the the section called ā€œCoding a Chart, Automaticallyā€). The only think I had to do was include the D3 library as instructed on the home page of D3 like so:
<script src="https://d3js.org/d3.v5.min.js"></script>
and then I copied the code from the notebook cell:

var data = [4, 8, 15, 16, 23, 42];

var x = d3.scaleLinear()
    .domain([0, d3.max(data)])
    .range([0, 600]);

const div=d3.select("body").append("div")
    .style("font", "10px sans-serif")
    .style("text-align", "right")
    .style("color", "white");

div.selectAll("div")
    .data(data)
    .join("div")
      .style("background", "steelblue")
      .style("padding", "3px")
      .style("margin", "1px")
      .style("width", d => `${d * 10}px`)
      .text(d => d);

and it worked!
You can see it running in codepen here
I hope that helps get you started.

This method should work for most D3 examples you see posted around the web, including some of the examples on Observable, except of course if the notebooks were also showing some interactivity and other features that Observable adds to using D3. But even then, in most cases you can add the bits you need together to create something similar in your web page.

There is also obviously the embed method that @mcmcclur described above, but it sounded like you just wanted to see an example of vanilla D3 in a web page. Hope this was helpful.

Good luck and have fun exploring D3!

3 Likes

I also believe that Observables only adds an obstacle to newcomers. Perhaps advanced users have found purpose in it, but personally I find itā€™s a major turn off at this point in time.

Hi Nathaniel,

Have a look at D3 for the Impatient. It uses plain old JavaScript and it reads quite nicely.

3 Likes

Your last paragraph hits the spot. Iā€™m learning d3js mostly from books and online tutorials. This was sufficient for some time but now even the latest book or tutorial isnā€™t up to date with the latest version of d3. Pretty much every book and tutorial uses the general update pattern which appears to be deprecated. It was replaced by sel.join() and if one desires to learn how to use the latter, it has to be through Observable. I canā€™t understand why even the basic functionality of d3 is tied to Observable. Iā€™m now left with two options, either learn Observable or keep using Canvas/JS for my projects. Iā€™m honestly not sure which way to go.

Props for your math animations, theyā€™re very inspiring!

3 Likes

Thanks! I literally published a new one on solving ordinary differential equations minutes ago:

I will say that, while I could have built this in a web page, I find the integration of graphics, text, and code for this kind of project to be quite natural in Observable.

2 Likes

Superb! Youā€™ve convinced me.

3 Likes

Similarly to another user here, I just created an account to express my opinion.

Iā€™m a Machine Learning Engineer, I do not care for front-end, but I like good looking visuals. Javascript is something I am trying to pick up as I go. I found d3js and thought it would be my savior for awesome visuals in published web posts.

I go to the official D3js webpage, it points me to observable. No where is it clear that Observable and D3js are in fact completely separate entitites. For my usecase, to build visuals running natively in the browser, I want Javascript. Not some abstraction layer on top of it.

If I should be constructive, its a bit of a catch 22, on the one hand improving the library as well as making it more accessible, or focusing solely on improving the library neglecting the aspects of accessibility.

I donā€™t know whats best, but it sure is frustrating to integrate observable gists into webpages as opposed to D3js.

3 Likes

ā€œI found $library, but all the examples lead me to CodePen/JSFiddle/CodeSandbox/$jssandbox, and itā€™s nowhere mentioned that $library and $jssandbox are two different things!ā€

They areā€¦

Besides, Observable is JS enough to just copy paste stuff into JS files, the only difference is that it allows people to experiment quickly and also write good documentation, the best D3 documentation Iā€™ve ever seen actually.

I got bad news though. D3 is not a visualisation framework, itā€™s building material. There will be no easy ā€œcopy paste this thing to get awesome visualsā€, not because of observable, but because of D3. Good custom visualisations are hard, lengthy, and time consuming, but they are worth all the sweat and tears.
If having a super nice live inspectable preview sandbox to experiment in is too much of a hurdle for you, then you wonā€™t fare well with D3 itself.

However if you want to learn D3 and JS as you go there is probably no better learning environment than Observable, so Iā€™d highly recommend taking the 30 minutes to learn it.

3 Likes

Adding to the general sentiment here: Iā€™m a software engineer, trying to build dashboards and charts for our product, deeply integrated in the corporate design, frontend framework and deployment process.
As much as I appreciate Observable for being an awesome, interactive playground for data, I donā€™t have the slightest intention of integrating an interactive playground in our product: I want to use D3, not Observable, because Iā€™m a developer in need of a library, not a data scientist in need of a visualization platform.
I want to show visuals of non-public KPIs and stats to our customers, in a form I decide on, without displaying this data to unauthorized users. I want to build lean and fast web apps without any additional runtime from Observable, adding overhead I donā€™t want or need. I want to limit the number of dependencies of my apps so I donā€™t have to monitor them for updates or license changes impacting my own.

And most of all, I would like to have a usable and up-to-date documentation of the visualization library I use, without spending huge amounts of paid developer time to convert Observable examples to Typescript. Iā€™m not complaining about the lack of official Typescript examples ā€” Thatā€™s a requirement I set for my own team and I donā€™t expect this level of commitment from others. But having so many subtle pitfalls in the examples, like reliance on the Observable standard library, makes D3 a nightmare to get started with.
The examples on bl.ocks.org and other websites and blogs are outdated to a large extent, relying on an old version of D3. Sometimes its enough to google for renamed methods, at other times the differences are too large to adapt examples to a new D3 version if youā€™re not too familiar with the library.

Take the Sankey chart, for example:

  • observablehq[dot]com/@d3/sankey-diagram?collection=@d3/d3-sankey (D3@6, d3-sankey@0.12)
  • observablehq[dot]com/[at]mbostock/flow-o-matic (D3@5, d3-sankey@0.12)
  • www.d3-graph-gallery[dot]com/graph/sankey_basic.html (D3@4, undisclosed custom sankey function)
  • bl.ocks[dot]org/wvengen/cab9b01816490edb7083 (D3@3, undisclosed old fork of sankey)
  • bl.ocks[dot]org/d3noob/c9b90689c1438f57d649 (D3@3, own sankey implementation)

All of these can be found among the first Google results for ā€œD3 Sankeyā€. Unless you know what youā€™re doing and what to watch out for, newbies have a high chance of getting it wrong, leading to frustration.

5 Likes

This comes across as ā€œI donā€™t intend to spend any of my time and money on your freely provided tool that is essential for my business. You should spend double the time and money catering precisely to my needs, so that I can exploit your work more efficiently. That you havenā€™t already is a ā€˜nightmareā€™.ā€

If you want the D3 volunteers to do thousands of hours of additional work to make your life easier, perhaps you could find some like-minded folks and take up a collection to pay them for the effort.

3 Likes

Well, no. I contribute quite a lot to different open source projects myself, itā€™s not that I take that volunteer effort for granted. To the contrary, I appreciate it immensely. But if you position a library as the go-to tool for all visualisation needs, it should be accessible to developers that want to use it outside of Observable or make it really clear that thatā€™s the intended use case.

Its not that Iā€”or other people on this thread, for that matterā€”ask for D3 to be modified to my wishes, but merely the documentation to be provided in a form that isnā€™t coupled to a separate platform with no easy way to use and tinker with code samples outside of said platform.
I strive for open source software to be as open and accessible as possible, and I donā€™t think itā€™s that outraging to request project maintainers to adhere to these principles, especially with seemingly nobody here acknowledging the fact that D3 is used outside of data science, too.

4 Likes

I created a sample notebook, where I describe how one can convert Observable specific d3.js code to vanilla javascript.

6 Likes

Wonderful. Thanks for explaining it simply in that notebook.

2 Likes

Hi

For me this thread illustrates two things that are conflated:

  • The audience as serious coders vs people (like me) who want to get into dataviz but are not completely competent in coding, and the already stated issue that
  • ā€œtying the D3 documentation too tightly to Observableā€ is a problem.

Thatā€™s the TL;DR.

I am a complete newbie to all things coding related. I came at this from a completely unrelated field (biology). I have tried for a few years, unsuccessfully, to learn D3 because I think it is the best way to visualise material flows (for urban metabolism, nutrient management and that sort of thing, that I used to research). I love Observable, because it finally allowed to me to see step by step all the things I was doing wrong, because it works in cells. I was doing sooooo many things wrong that the standard tutorials and ways to learn was not helping - you know when you misspell something so badly that spell checker canā€™t even help? That was me. I am now becoming much better at this, and starting to learn JavaScript and slowly getting back to the more pure coding stuff. I think this is a great platform, but you need to know why it exists for. I even think I can tackle Python again after this. I know these things are powerful but they were really just too much for me before. My main problem at the moment is that I get too distracted trying new things with my newfound power.

I donā€™t think Observable exists for serious coders. I think it is a nice notebook for some serious coders who want to quickly try something, or share a tutorial or example type of thing. If you ā€œalready know what interactive data visualization means and have already developed websitesā€ then, as I understand it, Observable is not primarily for you. I consider Observable to be ā€œa library as the go-to tool for all visualisation needs,ā€ BUT mainly for non-coding people.

For me data viz is going to be fundamentally frustrating and have a steep learning curve, because of the multidisciplinary nature of it. Journalism, data wrangling, statistics, story telling, graphic design, etc etc etc. So assuming that Observable, that exists to help with this, wonā€™t have some steep learning curve somewhere, is naive and unfair. Itā€™s also very new, so things will get better. Be kinder with your feedback.

I think dataviz is only going to be a bigger part of how we work and report, and Observable is helping to grow that competency base. I think it is a good platform for newcomers to code in general, rather than newcomers from code to D3, because it ā€œallows people to experiment quickly and also write good documentationā€ and the rest of j-pbā€™s reply.

What I am trying to do is play in Observable, and then export the thing I created as a picture. I am most excited by the interactive, animation stuff, and I think Iā€™ll have to learn a bit more about how to do that and export it to run on my website, for example. But as something to show to my colleagues / deliver to future clients to put excellently informative custom images in their reports, Observable is a win.

Lastly, this thread itself was very valuable, thank you. I realised there are things I struggle with that is not my incompetence but the D3-observable tightness, for example :slight_smile: And I learnt I can download the raw code!

5 Likes

You know, for all the discussion about this topic this is first concise, clear example of how to take a D3 example from Observable and run it. However, you may not appreciate a few of the subtle points I see illustrated here. I write this as a reasonably capable coder who has tried a few times to do something like this and got frustrated.

Hereā€™s what I got I hadnā€™t grasped before:

  1. all the code on an Observable page can be thought of as a single script. Your example takes data from the very top, the linear scale from further down and the last two bits from a single block.

  2. each of these Observable ā€˜blocksā€™ (is that the right term?) are functions - so each one has a return. That isnā€™t part of D3. Obvious once you know it. I didnā€™t understand it before.

The example page may have changed since you made your demo. As of today (10/6/2020) it uses

const div = d3.create("div")
  .style("font", "10px sans-serif")
  .style("text-align", "right")
  .style("color", "white");

But your CodePen uses:

const div = d3.select("body").append("div")
.style("font", "10px sans-serif")
.style("text-align", "right")
.style("color", "white");

Thatā€™s kind of an important point, it seems. Pasting the code from the current example Observable doesnā€™t work and doesnā€™t throw an error. So they arenā€™t equivalent statements.

  1. but there are other things too

This actually makes the point a lot of people keep bringing up - that itā€™s not nearly as straightforward to take examples of D3 from Observable, paste them into a real world project and have them work. Iā€™m not clear if you made the edit in CodePen or the source has changed since you made the demo. Either way itā€™s a show-stopper not knowing why the demo works but fails if I followed the same implied actions.

But to return to the lede - this is an excellent start. You give the original Observable you are working from, you show the example in CodePen and anyone interested can study it and figure out what they need to do.

I think thatā€™s the message a lot of posts have been sending. I think thatā€™s the way a lot of folks work. From this sort of example I might be interested to learn more about Observable because I can see what itā€™s doing for me instead of being frustrated by what itā€™s doing for me.

1 Like

Appreciate this thread, share a lot of the sentiments expressed. I have a little tweak on the original question for folks with some years of d3 experience:

Iā€™m learning Js and d3 at the same time, and a few months in. My gut told me to stick to writing everything in plain old Js after a similar ā€˜learning misfireā€™ with d3 and a prepared environment.

Folks with more experience ā€“ Is Observable the best place for keeping up with current practices? Iā€™ve been accessing what I thought were great resources only to find out on Observable that they were outdated. thx

1 Like

Yes - absolutely. If you follow the programming practices outlined in the user manual you will be using best D3 practices, as well as good Javascript practices in general. This is largely because of the community of D3 users here on Observable - most notably Mike who is both CTO of Observable, as well as the creator and lead developer of D3.

On the contrary, I think heā€™s asking them to stop spending their free time actively making his and other developerā€™s lives more difficult. Yes, Observable may make first-time learning of the general D3 concepts and interactivity easier, but itā€™s too closely intertwined with the only official examples of D3 (and more decoupled from JS) which is making learning D3 without learning (or learning around) Observable more and more difficult.

I think the most basic sentiment of frustration is that D3 is great, and Observable might be great for learning, but Observable is completely extraneous to D3 and coupling it with learning D3 is only helpful if you intend to continue with Observable. Most developers, particularly in industry, have no need for Observable and itā€™s a serious hindrance to learning D3, which, presumably, is why anyone ends up on Observable in the first place.

As a developer, I feel trapped. At this point, it been made clear that Observable is the direction that D3 is deliberately going, and D3 is currently an industry standard thatā€™s being actively obfuscated by the creators if you choose not to use their entire platform. This thread has convinced me that my choices are to 1) get out before it gets worse, 2) learn Observable well enough that I can decrypt it for my own purposes, or 3) accept the new environment and all that it will entail going forward. The latter is my least favorite option to say the least.

1 Like