Vega lite api miss-behaving

Mahesha999 · May 1, 2022, 8:42am

I prepared this vegalite visualization:

data = [{"student_name": "student 0", "e": "100.15", "d": "127.81"}, {"student_name": "student 1", "e": "100.30", "d": "189.94"}, {"student_name": "student 2", "e": "100.15", "d": "105.33"}, {"student_name": "student 3", "e": "99.41", "d": "85.36"}, {"student_name": "student 4", "e": "100.00", "d": "203.70"}, {"student_name": "student 5", "e": "100.15", "d": "139.05"}, {"student_name": "student 6", "e": "41.72", "d": "41.72"}, {"student_name": "student 7", "e": "100.30", "d": "260.50"}, {"student_name": "student 8", "e": "92.60", "d": "94.53"}, {"student_name": "student 9", "e": "99.11", "d": "86.39"}, {"student_name": "student 10", "e": "97.49", "d": "92.46"}, {"student_name": "student 11", "e": "98.37", "d": "83.58"}, {"student_name": "student 12", "e": "100.15", "d": "182.40"}, {"student_name": "student 13", "e": "100.15", "d": "99.41"}, {"student_name": "student 14", "e": "86.69", "d": "81.95"}, {"student_name": "student 15", "e": "98.08", "d": "306.51"}, {"student_name": "student 16", "e": "100.15", "d": "86.54"}, {"student_name": "student 17", "e": "100.15", "d": "186.39"}, {"student_name": "student 18", "e": "99.26", "d": "93.64"}, {"student_name": "student 19", "e": "100.15", "d": "102.66"}, {"student_name": "student 20", "e": "95.71", "d": "52.96"}, {"student_name": "student 21", "e": "99.85", "d": "99.41"}, {"student_name": "student 22", "e": "98.96", "d": "100.44"}, {"student_name": "student 23", "e": "100.15", "d": "131.07"}, {"student_name": "student 24", "e": "99.56", "d": "76.92"}, {"student_name": "student 25", "e": "100.15", "d": "213.46"}, {"student_name": "student 26", "e": "100.15", "d": "311.24"}, {"student_name": "student 27", "e": "100.15", "d": "21.89"}, {"student_name": "student 28", "e": "96.60", "d": "6.36"}, {"student_name": "student 29", "e": "53.70", "d": "3.70"}, {"student_name": "student 30", "e": "96.75", "d": "46.60"}, {"student_name": "student 31", "e": "100.15", "d": "100.15"}, {"student_name": "student 32", "e": "100.30", "d": "115.68"}, {"student_name": "student 33", "e": "87.13", "d": "103.85"}, {"student_name": "student 34", "e": "100.15", "d": "104.14"}, {"student_name": "student 35", "e": "99.26", "d": "59.17"}, {"student_name": "student 36", "e": "100.15", "d": "171.30"}, {"student_name": "student 37", "e": "99.11", "d": "94.08"}, {"student_name": "student 38", "e": "81.66", "d": "57.40"}, {"student_name": "student 39", "e": "96.01", "d": "154.59"}, {"student_name": "student 40", "e": "98.96", "d": "79.29"}, {"student_name": "student 41", "e": "99.26", "d": "118.20"}, {"student_name": "student 42", "e": "100.15", "d": "130.92"}, {"student_name": "student 43", "e": "90.53", "d": "47.63"}, {"student_name": "student 44", "e": "100.15", "d": "146.75"}, {"student_name": "student 45", "e": "96.01", "d": "197.63"}, {"student_name": "student 46", "e": "100.15", "d": "100.15"}, {"student_name": "student 47", "e": "107.25", "d": "115.24"}, {"student_name": "student 48", "e": "98.67", "d": "48.37"}, {"student_name": "student 49", "e": "88.61", "d": "190.53"}, {"student_name": "student 50", "e": "100.30", "d": "104.44"}, {"student_name": "student 51", "e": "51.18", "d": "51.18"}, {"student_name": "student 52", "e": "93.64", "d": "19.08"}, {"student_name": "student 53", "e": "100.00", "d": "123.08"}, {"student_name": "student 54", "e": "98.67", "d": "103.25"}, {"student_name": "student 55", "e": "98.37", "d": "95.86"}, {"student_name": "student 56", "e": "100.30", "d": "129.44"}, {"student_name": "student 57", "e": "100.15", "d": "164.50"}, {"student_name": "student 58", "e": "100.15", "d": "340.24"}, {"student_name": "student 59", "e": "100.30", "d": "165.83"}, {"student_name": "student 60", "e": "100.15", "d": "203.99"}, {"student_name": "student 61", "e": "92.75", "d": "160.06"}, {"student_name": "student 62", "e": "99.26", "d": "58.28"}, {"student_name": "student 63", "e": "99.41", "d": "99.41"}, {"student_name": "student 64", "e": "50.30", "d": "50.30"}, {"student_name": "student 65", "e": "93.79", "d": "50.30"}, {"student_name": "student 66", "e": "100.15", "d": "178.40"}, {"student_name": "student 67", "e": "99.11", "d": "6.21"}, {"student_name": "student 68", "e": "100.15", "d": "211.09"}, {"student_name": "student 69", "e": "100.15", "d": "69.08"}, {"student_name": "student 70", "e": "97.63", "d": "110.50"}, {"student_name": "student 71", "e": "100.15", "d": "126.33"}, {"student_name": "student 72", "e": "97.34", "d": "78.85"}, {"student_name": "student 73", "e": "100.15", "d": "182.99"}, {"student_name": "student 74", "e": "97.34", "d": "53.99"}, {"student_name": "student 75", "e": "98.22", "d": "45.12"}, {"student_name": "student 76", "e": "100.00", "d": "136.69"}, {"student_name": "student 77", "e": "98.82", "d": "41.72"}, {"student_name": "student 78", "e": "96.89", "d": "77.37"}, {"student_name": "student 79", "e": "97.78", "d": "81.36"}, {"student_name": "student 80", "e": "98.96", "d": "146.30"}, {"student_name": "student 81", "e": "99.26", "d": "48.67"}, {"student_name": "student 82", "e": "100.30", "d": "205.33"}, {"student_name": "student 83", "e": "98.82", "d": "141.42"}, {"student_name": "student 84", "e": "97.78", "d": "128.99"}, {"student_name": "student 85", "e": "100.15", "d": "86.69"}, {"student_name": "student 86", "e": "95.86", "d": "121.15"}, {"student_name": "student 87", "e": "97.49", "d": "88.31"}, {"student_name": "student 88", "e": "55.77", "d": "56.80"}, {"student_name": "student 89", "e": "96.01", "d": "13.17"}, {"student_name": "student 90", "e": "1.04", "d": "1.33"}, {"student_name": "student 91", "e": "99.70", "d": "26.18"}, {"student_name": "student 92", "e": "96.30", "d": "78.11"}, {"student_name": "student 93", "e": "99.85", "d": "11.83"}, {"student_name": "student 94", "e": "100.15", "d": "172.93"}, {"student_name": "student 95", "e": "100.00", "d": "198.82"}, {"student_name": "student 96", "e": "100.15", "d": "155.92"}, {"student_name": "student 97", "e": "92.01", "d": "97.19"}, {"student_name": "student 98", "e": "98.52", "d": "71.30"}, {"student_name": "student 99", "e": "100.15", "d": "111.69"}, {"student_name": "student 100", "e": "0.30", "d": "0.30"}, {"student_name": "student 101", "e": "91.72", "d": "86.54"}, {"student_name": "student 102", "e": "99.11", "d": "113.76"}, {"student_name": "student 103", "e": "98.22", "d": "67.75"}, {"student_name": "student 104", "e": "95.86", "d": "124.56"}, {"student_name": "student 105", "e": "105.92", "d": "201.48"}, {"student_name": "student 106", "e": "66.72", "d": "69.23"}, {"student_name": "student 107", "e": "100.15", "d": "151.18"}, {"student_name": "student 108", "e": "98.96", "d": "46.15"}, {"student_name": "student 109", "e": "100.15", "d": "160.36"}, {"student_name": "student 110", "e": "98.82", "d": "69.67"}, {"student_name": "student 111", "e": "98.08", "d": "80.03"}, {"student_name": "student 112", "e": "97.63", "d": "45.27"}, {"student_name": "student 113", "e": "96.60", "d": "37.28"}, {"student_name": "student 114", "e": "100.15", "d": "285.06"}, {"student_name": "student 115", "e": "98.96", "d": "77.81"}, {"student_name": "student 116", "e": "100.15", "d": "104.14"}, {"student_name": "student 117", "e": "100.15", "d": "160.65"}, {"student_name": "student 118", "e": "80.47", "d": "80.47"}, {"student_name": "student 119", "e": "100.15", "d": "115.09"}, {"student_name": "student 120", "e": "50.44", "d": "50.44"}, {"student_name": "student 121", "e": "100.15", "d": "136.83"}, {"student_name": "student 122", "e": "93.34", "d": "53.99"}, {"student_name": "student 123", "e": "100.15", "d": "136.09"}, {"student_name": "student 124", "e": "100.15", "d": "126.78"}, {"student_name": "student 125", "e": "89.35", "d": "74.85"}, {"student_name": "student 126", "e": "100.15", "d": "113.17"}, {"student_name": "student 127", "e": "97.78", "d": "78.25"}, {"student_name": "student 128", "e": "94.67", "d": "28.25"}, {"student_name": "student 129", "e": "100.15", "d": "154.88"}, {"student_name": "student 130", "e": "80.47", "d": "101.78"}, {"student_name": "student 131", "e": "99.11", "d": "66.42"}, {"student_name": "student 132", "e": "103.40", "d": "260.50"}, {"student_name": "student 133", "e": "99.11", "d": "17.75"}, {"student_name": "student 134", "e": "86.98", "d": "89.94"}, {"student_name": "student 135", "e": "96.30", "d": "81.66"}, {"student_name": "student 136", "e": "100.15", "d": "60.36"}, {"student_name": "student 137", "e": "100.15", "d": "73.52"}, {"student_name": "student 138", "e": "96.45", "d": "68.93"}, {"student_name": "student 139", "e": "100.30", "d": "165.83"}, {"student_name": "student 140", "e": "96.89", "d": "55.77"}, {"student_name": "student 141", "e": "100.15", "d": "112.72"}, {"student_name": "student 142", "e": "99.26", "d": "204.88"}, {"student_name": "student 143", "e": "100.15", "d": "356.51"}, {"student_name": "student 144", "e": "99.70", "d": "152.66"}, {"student_name": "student 145", "e": "100.15", "d": "40.83"}, {"student_name": "student 146", "e": "92.16", "d": "63.46"}, {"student_name": "student 147", "e": "100.15", "d": "179.88"}, {"student_name": "student 148", "e": "100.00", "d": "81.36"}, {"student_name": "student 149", "e": "16.57", "d": "11.98"}, {"student_name": "student 150", "e": "84.62", "d": "73.08"}, {"student_name": "student 151", "e": "100.59", "d": "100.59"}, {"student_name": "student 152", "e": "98.82", "d": "104.88"}, {"student_name": "student 153", "e": "99.11", "d": "135.95"}, {"student_name": "student 154", "e": "97.93", "d": "35.95"}, {"student_name": "student 155", "e": "100.15", "d": "101.92"}, {"student_name": "student 156", "e": "100.30", "d": "120.86"}, {"student_name": "student 157", "e": "81.07", "d": "88.76"}, {"student_name": "student 158", "e": "100.15", "d": "128.40"}, {"student_name": "student 159", "e": "94.67", "d": "119.23"}, {"student_name": "student 160", "e": "100.30", "d": "253.85"}, {"student_name": "student 161", "e": "91.42", "d": "68.05"}, {"student_name": "student 162", "e": "100.15", "d": "118.79"}, {"student_name": "student 163", "e": "100.15", "d": "101.04"}, {"student_name": "student 164", "e": "100.30", "d": "203.11"}, {"student_name": "student 165", "e": "100.59", "d": "155.18"}, {"student_name": "student 166", "e": "96.89", "d": "84.76"}, {"student_name": "student 167", "e": "100.15", "d": "115.38"}, {"student_name": "student 168", "e": "100.00", "d": "179.73"}, {"student_name": "student 169", "e": "100.15", "d": "135.65"}, {"student_name": "student 170", "e": "98.37", "d": "93.34"}, {"student_name": "student 171", "e": "100.15", "d": "128.40"}, {"student_name": "student 172", "e": "100.44", "d": "164.05"}, {"student_name": "student 173", "e": "98.37", "d": "57.54"}, {"student_name": "student 174", "e": "96.75", "d": "118.05"}, {"student_name": "student 175", "e": "98.96", "d": "91.12"}, {"student_name": "student 176", "e": "100.00", "d": "226.04"}, {"student_name": "student 177", "e": "98.67", "d": "150.89"}, {"student_name": "student 178", "e": "97.49", "d": "68.79"}, {"student_name": "student 179", "e": "100.15", "d": "133.58"}, {"student_name": "student 180", "e": 0, "d": 0}, {"student_name": "student 181", "e": 0, "d": 0}, {"student_name": "student 182", "e": 0, "d": 0}, {"student_name": "student 183", "e": 0, "d": 0}, {"student_name": "student 184", "e": 0, "d": 0}, {"student_name": "student 185", "e": 0, "d": 0}, {"student_name": "student 186", "e": 0, "d": 0}, {"student_name": "student 187", "e": 0, "d": 0}, {"student_name": "student 188", "e": 0, "d": 0}, {"student_name": "student 189", "e": 0, "d": 0}, {"student_name": "student 190", "e": 0, "d": 0}, {"student_name": "student 191", "e": 0, "d": 0}, {"student_name": "student 192", "e": 0, "d": 0}, {"student_name": "student 193", "e": 0, "d": 0}, {"student_name": "student 194", "e": 0, "d": 0}, {"student_name": "student 195", "e": 0, "d": 0}, {"student_name": "student 196", "e": 0, "d": 0}, {"student_name": "student 197", "e": 0, "d": 0}, {"student_name": "student 198", "e": 0, "d": 0}, {"student_name": "student 199", "e": 0, "d": 0}, {"student_name": "student 200", "e": 0, "d": 0}, {"student_name": "student 201", "e": 0, "d": 0}, {"student_name": "student 202", "e": 0, "d": 0}, {"student_name": "student 203", "e": 0, "d": 0}, {"student_name": "student 204", "e": 0, "d": 0}, {"student_name": "student 205", "e": 0, "d": 0}, {"student_name": "student 206", "e": 0, "d": 0}, {"student_name": "student 207", "e": 0, "d": 0}, {"student_name": "student 208", "e": 0, "d": 0}, {"student_name": "student 209", "e": 0, "d": 0}, {"student_name": "student 210", "e": 0, "d": 0}, {"student_name": "student 211", "e": 0, "d": 0}]

createChart = function (data) {
  max_d = d3.max(data, record => parseFloat(record.d));
  max_e = d3.max(data, record => parseFloat(record.e));
  max_y_scale_value_for_d = d3.max([100, max_d]);
  max_y_scale_value_for_e = d3.max([100, max_e]);

  const mpg = vl.markLine()
              .data(data)
              .transform([{ "calculate": "toNumber(datum.d)", "as": "d2" }, { "calculate": "toNumber(datum.e)", "as": "e2" }])
			  
              .encode(
                      vl.x()
                        .fieldN('student_name')
                        .title('students')
                        .axis({ labels: false, ticks: false })
                        .sort('-y'),
                      vl.y()
                        .fieldQ('d2')
                        .scale({ "domain": [0, max_y_scale_value_for_d] })
                        .title('D'),		
                      vl.tooltip([{ 'field': 'student_name', 'title': 'Sudent Name' },
                                  { 'field': 'd', 'title': 'D' },
                                  { 'field': 'e', 'title': 'E' }])
                                ).width(500).height(250);

  const hp = mpg.markLine({ color: '#227a15' })		
              .encode(
                      vl.y()
                        .fieldQ('e2')
                        .scale({ "domain": [0, max_y_scale_value_for_e] })
                        .title('E')
                     );

  const plot = vl.vconcat(
                       vl.layer(mpg, mpg.markCircle()),
                       vl.layer(hp, hp.markCircle({ color: '#227a15' }))
                     )

  return plot.toObject();
}

const chart_spec_json = this.createChart(data)
const opt = {
               renderer: "canvas",
               actions: false
            };
vegaEmbed("#stats", chart_spec_json, opt);

Run the above code to see how it gets rendered. This is how it looks. (Sorry forum is not allowing me to have more than one image embedded.)

However, when I do the exact same with the same code and the same data (with only real student names), with same version of dependencies loaded in my development environment, it gets rendered like this:

Notice the weird horizontal spike in the line charts. It seems that one student is not getting correctly sorted in the order. But why this might be happening?

Mahesha999 · May 1, 2022, 8:43am

You can check the example in action embedded in this stackoverflow question.

(I have to put this as a comment as forum did not allow to have more than one URL in the question.)

mcmcclur · May 1, 2022, 1:05pm

There are certainly a few folks here who know VegaLite very well, so you might get a better answer than this, but this really isn’t a forum for general Javascript questions. In particular, using something like vegaEmbed to embed into a webpage, is outside the scope of this forum.

Having said that, you can certainly use Observable to generate those charts, as shown in this notebook. You can also use Observable’s embedding features to embed those charts, like so:

Mahesha999 · May 1, 2022, 9:03pm

Well, actually I tried the same visualization in Observable already here. It renders correctly. I used vega-embed to embed it in stackoverflow post, which I copy pasted in above original question. I was unaware of observable embed function. Also used vega-embed in out project also as I cannot find other option to render vega lite api output.

I didnt get what exactly you were saying:

Do you mean to say I can do away with vega-embed in original question above?
Do you mean to say vega-embed can be the culprit in the visualization glitch in my development environment? (But then it gets rendered correctly in the stackoverflow)

mcmcclur · May 1, 2022, 9:29pm

When asking questions here, it’s usually a good idea to include a pointer to a notebook with your work, if you’ve got one.

Perhaps. There might be good reasons to use vega-embed in your workflow, like your accustomed to it or you’re working with other folks who arre already using it. But, yes, I also think that embedding from Observable is a viable approach - in fact, the preferred approach by folks who use this forum.

Well, I don’t know anything at all about vega-embed and, really, I don’t think this forum is a good place to be asking about vega-embed.

mootari · May 2, 2022, 9:36am

Then it’s not the same data, though? Try filtering/modifying your dataset until the error goes away. This should allow you to narrow down the offending item.

Note that you can fetch local data in an Observable notebook without having to upload it. Just start a local HTTP server to serve your JSON or CSV:

npx http-server --cors

Alternatively you can add a file input to temporarily attach your data while keeping it local.

Mahesha999 · June 4, 2022, 9:08am

@mootari @mcmcclur Finally I am able to replicate the issue.
It happens when there are two students with one non zero value and one zero value. Below student134 has these values:

{"student_name": "student 134", "e": 0, "d": 0}
{"student_name": "student 134", "e": "86.98", "d": "89.94"}

and leads to this spike:

This also happens with same student with two non zero values:

Why is this so? Cant there be two students with same name? Is this a bug or did I miss some vega spec config?

mootari · June 6, 2022, 8:04pm

Your x axis is a nominal axis based on student names, so this behavior seems to be expected? You’d probably at least need a second sort criteria to handle the ill defined cases (e.g. multiple zero values), but you may want to use the data index instead and just refer to the name for the labels (if possible).

Note also that it helps a lot to reduce your example to the bare minimum that is necessary to understand the problem (both as a courtesy and to limit noise). In your case, the dataset can be reduced to

data2 = [
  { student_name: "student 1", e: 100.15, d: 171.30 },
  { student_name: "student 134", e: 86.98, d: 89.94 },
  { student_name: "student 156", e: 100.30, d: 120.86 },
  { student_name: "student 157", e: 81.07, d: 88.76 },
  { student_name: "student 192", e: 0, d: 0 },
  { student_name: "student 193", e: 0, d: 0 },
  { student_name: "student 134", e: 0, d: 0 }
]

and the relevant Vega-Lite code to

test1234 = {
  const mpg = vl
    .markLine()
    .data(data2)
    .encode(vl.x().fieldN("student_name").sort("-y"))
    .width(500);

  const hp = mpg.markLine().encode(vl.y().fieldQ("e"));

  return vl.layer(hp).render();
}

Topic		Replies	Views
Changing vega lite markLine to markBar makes it lose sort order Community Help	1	308	May 24, 2022
code formatting [vega-lite] Community Help	3	522	April 28, 2020
Parsing dynamic variable in vegalite api Community Help	3	513	November 28, 2020
Vega_lite_Disable Blank Facet Community Help	3	768	October 20, 2021
Help on Vega-Lite plot Community Help	3	543	September 6, 2019

Vega lite api miss-behaving

Related topics