# Integrating ganja.js - thoughts

**URL:** <https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651>\
**Category:** Help\
**Created:** [April 16, 2018, 8:40pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651 "2018-04-16T20:40:21Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 16, 2018, 8:40pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/1 "2018-04-16T20:40:21Z")

</div>

Hi I uploaded my library (ganja.js) to npm so I could include it more easily in observablehq. Getting an error now without line numbers or context - anyway I could get to see what I’m doing wrong ? (works fine in node, the browser and npm’s preview tool)

```javascript
Algebra = import("ganja.js@1.0.3/ganja.js")

```

Also, ganja.js uses DOMparser in one of its (browser targetted) functions to parse an SVG string - but in observablehq this interface is not available ?

Apologies for the noob questions - thanks in advance for any help.

---

<div class="post-metadata">

**Author:** ![Fil](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/fil/32/207_2.png) [@Fil](https://talk.observablehq.com/u/Fil)\
**Post date:** [April 16, 2018, 8:44pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/2 "2018-04-16T20:44:18Z")

</div>

A lot of people will be happy to see you!

A trick I use to test my modules is to load them locally:  
`Algebra =` ~~import~~ require`("http://localhost/ganja.js/build/ganja.js")`

strangely it works (from Chrome at least) which allows all sorts of dirty debugging with console.log or other tools.

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 16, 2018, 8:45pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/3 "2018-04-16T20:45:47Z")

</div>

Ah excellent - should’ve thought of that 🙂 - trying now - thanks Fil !

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 16, 2018, 9:14pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/4 "2018-04-16T21:14:48Z")

</div>

Ok - if I use require instead of import - it loads fine 👍

```javascript
Algebra = require("ganja.js/ganja.js")

```

Next question - perhaps I should start a new topic … my graph function returns either a canvas or an svg object depending on what it needs to graph. If I return a canvas object it works fine …

```javascript
Algebra(0,()=>this.graph(x=>Math.sin(x*Math.PI)/2))

```

It gets displayed correctly as the cell output.

if however I return an svg object, it simply shows SVGSVGElement … no image.

I created a quick [notebook to demonstrate](https://beta.observablehq.com/@enkimute/svg-element-not-showing)

Should I not expect the SVG element to be displayed as image ?

any help appreciated !

---

<div class="post-metadata">

**Author:** ![Fil](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/fil/32/207_2.png) [@Fil](https://talk.observablehq.com/u/Fil)\
**Post date:** [April 16, 2018, 9:30pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/5 "2018-04-16T21:30:34Z")

</div>

sorry I should have spotted this.

Your example can be fixed with

```auto
{
  var div = DOM.element("div");
  div.appendChild( Algebra(2,0,1,()=>this.graph([1e12,"o",1e1,"x",1e2,"y"])) );
  return div;
}

```

---

<div class="post-metadata">

**Author:** ![tom](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/tom/32/2262_2.png) [@tom](https://talk.observablehq.com/u/tom)\
**Post date:** [April 16, 2018, 9:31pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/6 "2018-04-16T21:31:02Z")

</div>

The reason for that is something I demonstrate in [this notebook](https://beta.observablehq.com/@tmcw/using-ganja-js#line) - Ganga appends the SVG element to the page by itself as well as returns it as a value. Observable can’t show the same element in two places, so the only way to cleanly display it is to detach it, or as Fil points out, attach it to a different container.

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 16, 2018, 9:56pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/7 "2018-04-16T21:56:18Z")

</div>

Hi Tom,

That´s a cool notebook !

ganja.js does not append to the page. The graph function only returns the svg element. A simplified testcase is this :

```javascript
(function(){
  return new DOMParser().parseFromString(`<svg height="100" width="100"><circle cx="50" cy="50" r="40" stroke="black" stroke-width="3" fill="red"/></svg> `,'text/html').body.firstChild;
})()

```

I would expect that to simply return the svg element - you can paste that in the console - it seems to do just that. (setting innerHTML on svg elements does not work - which is why I’m using DOMParser).

Although pasting that in observable gives an error on DOMParser being not defined.  
(Which I saw earlier when trying to include ganja, but using require seems to have resolved that to)

Again, sorry for the noob stuff - I’ve only skimmed through the intros - I really should dive into the docs first.

Once I get past these basics I plan on thinking about what we can do to better combine the idea of transpiled Javascript and observablehq. (as you and others have noticed, you lose some lexical scoping and I want to play around to see if there’s a workaround possible - or which model would fit best if not)

Thanks for the quick reply,

Cheers !

(btw I’ve also made a notebook with an easier introduction on ganja.js - some simple complex numbers and funny array stuff … [in case you haven’t seen it yet](https://beta.observablehq.com/@enkimute/ganja-js-introduction))

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 16, 2018, 10:05pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/8 "2018-04-16T22:05:39Z")

</div>

The [_domParser_.parseFromString](https://developer.mozilla.org/en-US/docs/Web/API/DOMParser) call returns a new document with a document element and a body. So from the perspective of Observable, the returned SVG element is already attached to a document—it just so happens to be a different document than the main one. You need to remove the SVG element from its parent before returning it from the cell.

Observable does not append an element that already has a parent because doing so would remove it from its existing parent; this is a destructive side effect and we generally try to avoid side effects. Observable only appends detached elements.

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 16, 2018, 10:27pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/9 "2018-04-16T22:27:44Z")

</div>

Gotcha !

github & npm are updated … ganja.js can now cleanly be included :

```javascript
Algebra = require('ganja.js')

```

and all graphing works as expected

```javascript
Algebra(2,0,1,()=>this.graph([1e12,1e1,1e2]))

```

Thanks for the help !

👍👍👍

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 16, 2018, 10:49pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/10 "2018-04-16T22:49:36Z")

</div>

Right on! I look forward to more ganja.js notebooks! 😄

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 17, 2018, 8:09am UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/11 "2018-04-17T08:09:38Z")

</div>

Mike,

Have you got any ideas on how to better integrate transpiled javascript into observablehq ?

ganja.js translates javascript to javascript, keeping unknown keywords (say ‘mutable’) in place, so it has the potential of integrating very neatly.

Just freewheeling here, and I’m totally open to suggestions - but two options come to mind. One option would be a way to tell observable to transpile javascript … this would potentially enable smooth integration with other js to js transpilers.

Another option would be for observable to provide a ‘scoped eval’ call so that you could tell ganja to use that call to evaluate its translated functions.

The first option, where observable knows it needs to transpile javascript :

```javascript
observablehq.setTranspiler(Algebra(2,0,1).inline); // just freewheeling

```

after which cells could contain full math like so :

```javascript
xy = 1e12

```

followed by e.g.

```javascript
z = 1e123 * xy

```

or

```auto
even=[1,2,3,4,5,z]*2;

```

In the second scenario they would look like so :

```javascript
PGA2D = Algebra({p:2,r:1,eval:observablHQ.scopedEval})

```

```javascript
p = PGA2D.inline(()=>1e12)()

```

```javascript
z = PGA2D.inline(()=>1e123*xy)()

```

```javascript
even = PGA2D.inline(()=>[1,2,3,4,5,z]*2)()

```

Ofcourse, chances are you already have a better model in place, looking forward to hearing your thoughts.

---

<div class="post-metadata">

**Author:** ![kelleyvanevert](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/kelleyvanevert/32/169_2.png) [@kelleyvanevert](https://talk.observablehq.com/u/kelleyvanevert)\
**Post date:** [April 17, 2018, 8:29am UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/12 "2018-04-17T08:29:50Z")

</div>

Haha, that would be a bit crazy, but crazy cool 🙂

The whole lexical scoping issue for ‘inline functions’ can of course be swept under the rug from the perspective of an Observable notebook writer, because Observable ‘invents’ this scope outside of the single cell anyway.

I’m not sure if Tom/Jeremy/Mike are so keen on adding transpilers, but this would be a case where I’d just love it 🙂

---

<div class="post-metadata">

**Author:** ![jashkenas](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/jashkenas/32/1778_2.png) [@jashkenas](https://talk.observablehq.com/u/jashkenas)\
**Post date:** [April 17, 2018, 5:20pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/13 "2018-04-17T17:20:09Z")

</div>

Very intriguing — and especially your “matrix” multiplication example is a very compelling taste of just how useful an “include my transpiler” directive might be.

One thing I wanted to mention is that: Although we don’t yet have concrete plans about how to (or whether to) implement this sort of thing, we _are_ hoping to open-source the Observable transpiler (mutable, viewof, cell references), and the runtime in the near future. And once those pieces of the puzzle are available, maybe you folks can use them to explore some of the most interesting possibilities for pre-compilation of cell bodies…

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 17, 2018, 7:07pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/14 "2018-04-17T19:07:55Z")

</div>

Our current approach for transpilation (dialects or languages other than JavaScript) is based on tagged template literals. We do this for Markdown (`md`), HTML (`html`, `svg`) and LaTeX (`tex`).

You could do something similar for ganja. Sketching:

```auto
e2 = Algebra(2, 0, 1)

```

```auto
xy = e2`1e12`

```

```auto
z = e2`1e123 * ${xy}`

```

```auto
even = e2`[1, 2, 3, 4, 5, ${z}] * 2`

```

You’d lose the syntax highlighting, of course, and you’d probably need to reparse and regenerate the code when a reactive reference changes (_e.g._, _z_ in _even_), but maybe we can think of a clever way to avoid that, say using [this values](https://beta.observablehq.com/@tmcw/untitled/5).

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 17, 2018, 7:11pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/15 "2018-04-17T19:11:15Z")

</div>

To clarify a subtle point: the embedded expressions within the template literals needn’t be string-coerced. That is how, for example, you can embed a DOM element or an array of DOM elements within an `html` template literal. See the source here:

> <https://github.com/observablehq/notebook-stdlib/blob/master/src/template.js>

I imagine you could do something similar for ganja where the embedded expressions become placeholders (arguments) that you can pass in during evaluation.

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 18, 2018, 10:19am UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/16 "2018-04-18T10:19:42Z")

</div>

Hi Mike,

There’s a notebook [here](https://beta.observablehq.com/@enkimute/ganja-js-template-test) that give this a spin. Missing syntax highlighting is unfortunate, and for complex expressions, the extra ${} can add quite a bit of clutter.

the original ganja.js code :

```javascript
  var orthocenter = (B&C<<A)^(C&A<<B);

```

which calculates the orthocenter of a triangle given the three vertices A,B,C. It’s quite readable if you know PGA (from left to right : find the line orthogonal to the line BC and through A and intersect it with the line orthogonal to the line CA and through B). now looks like this if you want it in a separate cell :

```javascript
  orthocenter = PGA2D`(${B}&${C}<<${A})^(${C}&${A}<<${B})`

```

It all works but feels like a bit of a shame since a transpiler that works before observable would allow to drop all the clutter. For a project like ganja that is all about clean-lean-mean-math-syntax … I’ll keep hoping for a better way …

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 18, 2018, 3:12pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/17 "2018-04-18T15:12:48Z")

</div>

Thanks for exploring this idea. I agree the additional syntax for embedded expressions are cumbersome in this case (since the DSL is already so close to JavaScript).

Our tagged template literals allow arbitrary languages to be embedded in vanilla JavaScript. These literals are interpreted by their tag function after Observable transpiles the cell definition to vanilla JavaScript. It seems like what we want here is a transformation _prior_ to Observable transpiling the cell definition to JavaScript, rather than after. That way, Observable can parse the transformed cell definition and extract the references to build the dataflow graph.

So for example, say you have the following cell definition, and Observable somehow knows to apply a _PGA2D_ transformation:

```auto
orthocenter = (B&C<<A)^(C&A<<B)

```

The transformation would run, converting this into something like:

```auto
orthocenter = CC.Wedge(CC.Dot(CC.Vee(C, A), B), CC.Dot(CC.Vee(A, B), C))

```

This is now (Observable) JavaScript, and can be parsed and executed normally by the Observable runtime.

But there are several challenges to this approach:

- How do you define and register a transpiler?

- How do you declare that a cell should be transformed by a given transpiler? (I don’t think we want this operation to be global to the notebook; their either needs to be some sort of pragma or metadata associated with each cell that specifies the transpiler.)

- How do transpilers inject runtime values, such as the `CC` Algebra instance above? Are these injected values reactive?

- When a cell is transpiled, how does syntax highlighting and autocomplete work? (Possibly these are simply disabled when transpilation is active; otherwise we’d have to define extension mechanisms for these features, too.)

Ideally, the transpiler is defined in a cell, and maybe there’s a tiny bit of syntax to apply it to a cell, like:

```auto
PGA2D = (await require("ganja.js"))(2, 0, 1)

```

```auto
orthocenter = PGA2D: (B&C<<A)^(C&A<<B)

```

Anyway, this isn’t an easy problem, sorry! But enabling extensible transpilation could be a very powerful feature, so we’re going to keep thinking about it.

---

<div class="post-metadata">

**Author:** ![mbostock](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mbostock/32/9_2.png) [@mbostock](https://talk.observablehq.com/u/mbostock)\
**Post date:** [April 18, 2018, 3:24pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/18 "2018-04-18T15:24:09Z")

</div>

Another possibility is that the transpiler skips the Observable JavaScript representation and goes straight to vanilla JavaScript. Then the transpiler would be responsible for extracting the references from the code and returning an array of named inputs and a function body. (We’re going to open-source the runtime soon, which will make this API a bit more clear) As in:

```auto
({
  inputs: ["A", "B", "C"],
  value: (A, B, C) => CC.Wedge(CC.Dot(CC.Vee(C, A), B), CC.Dot(CC.Vee(A, B), C))
})

```

That lets the transpiler use closure for its runtime values (`CC`).

---

<div class="post-metadata">

**Author:** ![enkimute](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/enkimute/32/500_2.png) [@enkimute](https://talk.observablehq.com/u/enkimute)\
**Post date:** [April 18, 2018, 3:45pm UTC](https://talk.observablehq.com/t/integrating-ganja-js-thoughts/651/19 "2018-04-18T15:45:20Z")

</div>

Hi Mike,

Thanks for the feedback. The key is indeed that this type of transformation can only truly deliver its power if it sits before observable. Your assessment is spot-on. I’m fine with either option. (although ganja.js currently does not know what variables are from what scope and simply doesn’t touch any of them).

For now, I’ve upgraded ganja so that **inline** can also be used as template function - it makes the template approach as simple as I can get it :

```javascript
Algebra = require("ganja.js")

```

```javascript
p2 = Algebra(2,0,1).inline

```

```javascript
some_point = p2`1e12+.5e02`

```

```javascript
some_expression = p2`2*${some_point}`

```

For those libs that are true js-\>js (like ganja or say prepack) it would really be killer to be able to keep syntax highlighting. (ganja’s operators and syntax have been painfully restricted to keep editor features valid)

I have no brilliant suggestions for how that would be most elegantly specified in observable - one idea is to augment template string functions with extra properties. (i.e. specify for a template function what syntax highlighter to use, and if its a ‘before’ or ‘after’ template)

```javascript
var templateFunc = Algebra(2,0,1).inline;
templateFunc.syntaxHighlight = 'javascript';
templateFunc.beforeObservable = true;

```

This would give an ‘in’ to also do syntax highlighting on html,svg,tex, …

Just a thought.

Cheers,

S.