# Data table cell arrow data support

**URL:** https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737
**Category:** Help
**Created:** [September 4, 2024, 6:21am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737 "2024-09-04T06:21:53Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 4, 2024, 6:21am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/1 "2024-09-04T06:21:53Z")

</div>

I use `@uwdata/vgplot` duckdb-wasm to query data from http,  
if vgplot query returns array will accept by data table cell, but if query returns arrow, data table cell report this error:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/0/0e1de10c22f44f4490669e81ac3326c1bb632129.png)

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 4, 2024, 10:33am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/2 "2024-09-04T10:33:01Z")

</div>

Are you able to share your notebook (or a simplified example)?

---

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 9, 2024, 3:09am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/3 "2024-09-09T03:09:16Z")

</div>

this is an example: [Untitled / sn | Observable](https://observablehq.com/@cnsn/data-preview2)

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 9, 2024, 10:01am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/4 "2024-09-09T10:01:02Z")

</div>

Can you share your repro steps? I only manage to trigger the error every once in a while if I randomly select and deselect columns.

---

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 10, 2024, 7:30am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/5 "2024-09-10T07:30:57Z")

</div>

sorry, I had set the default data source to `vg_data_array_is_ok`, did you change the data cell source to `vg_data_arrow`, I meet this error all the time

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/a/a0e340b0dd8fc54fcd146aefb6ba38ece8319d57.png)

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 10, 2024, 8:52am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/6 "2024-09-10T08:52:27Z")

</div>

Looks like the file parquet file isn’t served correctly. I can see CORS errors in the network tab.

---

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 10, 2024, 10:18am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/7 "2024-09-10T10:18:57Z")

</div>

sorry, the CORS was revoked by IaC, please try again, or you can pass a file url parameter to override the default parquet file witch fit your network like this `https://observablehq.com/@cnsn/data-preview2?url=https//www.to/file.parquet`

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 10, 2024, 11:00am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/8 "2024-09-10T11:00:54Z")

</div>

Now I’m also getting the error consistently. I suspect that some race conditions and/or side effects might be at play. Perhaps you can try to model the dependencies more explicitely?

It looks like vg\_data\_arrow does not register as table:

```auto
(await DuckDBClient.of({vg_data_arrow, penguins})).sql`show tables`

```

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 10, 2024, 7:19pm UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/9 "2024-09-10T19:19:57Z")

</div>

After changing the type to `json` and playing around some more with other Arrow examples I can see now that this is likely a bug on our end.

I’ve filed an internal issue, but I’m afraid I don’t know how soon we’ll be able to fix it. 🙏

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 10, 2024, 7:40pm UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/10 "2024-09-10T19:40:21Z")

</div>

I suspect this is caused by

> <https://github.com/duckdb/duckdb-wasm/issues/1708>
>
> \### What happens?
> 
> The method \`Connection.insertArrowTable\` silently fails in a …typescript program.
> 
> \### To Reproduce
> 
> \## bare-node works
> 
> Change the query in \`examples/bare-node/index.cjs\` from:
> 
> \`\`\`js
> const conn = await db.connect();
> 
> await conn.query(\`SELECT count(\*)::INTEGER as v FROM generate\_series(0, 100) t(v)\`);
> 
> await conn.close();
> \`\`\`
> 
> to:
> 
> \`\`\`js
> const conn = await db.connect();
> 
> const json = \[
> { a: 1, b: 11 },
> { a: 2, b: 22 },
> \];
> 
> const table = arrow.tableFromJSON(json);
> await conn.insertArrowTable(table, {
> name: 'local\_table',
> create: true,
> });
> try {
> const result = await conn.query(\`SELECT \* FROM local\_table\`);
> console.log(result.toString());
> } catch (error) {
> console.log(error);
> }
> 
> await conn.close();
> \`\`\`
> 
> The table should be created and the query succeeds.
> 
> \## esbuild-node fails
> 
> Change the query in \`examples/esbuild-node/index.ts\` from:
> \`\`\`ts
> const conn = await db.connect();
> 
> await conn.query\<{ v: arrow.Int }\>(\`SELECT count(\*)::INTEGER as v FROM generate\_series(0, 100) t(v)\`);
> 
> await conn.close();
> \`\`\`
> 
> to:
> 
> \`\`\`ts
> const conn = await db.connect();
> 
> const json = \[
> { a: 1, b: 11 },
> { a: 2, b: 22 },
> \];
> 
> const table = arrow.tableFromJSON(json);
> await conn.insertArrowTable(table, {
> name: 'local\_table',
> create: true,
> });
> try {
> const result = await conn.query\<{ a: arrow.Int; b: arrow.Int }\>(\`SELECT \* FROM local\_table\`);
> console.log(result.toString());
> } catch (error) {
> console.log(error);
> }
> 
> await conn.close();
> \`\`\`
> 
> This should \*fail\*.
> 
> \## Investigation
> 
> When I step through the code I notice the failure is in \`apache-arrow/ipc/writer.js\` in the function \`writeAll\` at the statement \`if (input instanceof table\_js\_1.Table) {\`. In the \`bare-node\` example this is \`true\` while for \`esbuild-node\` this is \`false\`.
> Furthermore for \`bare-node\` \`input.\_\_proto\_\_ === table\_js\_1.Table.prototype\` is \`true\`, but for \`esbuild-node\` it is \`false\`.
> 
> It appears the internal version of the \`apache-arrow\` \`Table\` class is different from that of the locally imported one. 
> 
> I note the \`bundle.mjs\` in \`duckdb-wasm\` patches the \`appache-arrow/package.json\` due to the export strategy of the \`appache-arrow\` package. Could this be involved?
> 
> \### Browser/Environment:
> 
> Chrome 123.0.6312.123 
> 
> \### Device:
> 
> MacBook Pro (ARM)
> 
> \### DuckDB-Wasm Version:
> 
> Latest clone (also 1.28.1-dev179.0)
> 
> \### DuckDB-Wasm Deployment:
> 
> The duckdb-wasm repo
> 
> \### Full Name:
> 
> Rob Blackbourn
> 
> \### Affiliation:
> 
> None

---

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 11, 2024, 2:21am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/11 "2024-09-11T02:21:17Z")

</div>

I find that `Input.table` can accept `vg_data_arrow` correctly:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/observablehq/original/2X/8/896af7f0ad172526886cf9189e802a08a8267403.png)

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 11, 2024, 10:02am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/12 "2024-09-11T10:02:00Z")

</div>

Data table cells use DuckDB under the hood, so they inherit some of the bugs. Inputs.table() simply requires an Iterable of objects and does not look at the schema at all.

As a workaround you can use

```js
vg_data_array = vg.coordinator().query(sql, {type: "json", /* ... */})

```

(By the way, you don’t need `await` at the top level - Observable’s Runtime automatically awaits Promises returned by cells.)

As mentioned, the underlying problem is that three different module instances (and versions) of Apache Arrow are being loaded. Mosaic uses a different arrow module instance to produce the Arrow Table than Observable’s Stdlib, which then fails DuckDB’s instance checks. Observable’s DuckDBClient.of correctly detects Arrow Tables regardless of version because it uses duck typing (no pun intended), but DuckDB itself will then fail to detect the Table and skips it.

To give an example, try the following:

```js
// Import the same arrow module URL as used by Observable's DuckDB - modules are singletons
arrow11 = import("https://cdn.observableusercontent.com/npm/apache-arrow@11.0.0/+esm")

```

```js
// Convert to an arrow 11 Table
vg_data_arrow = arrow11.tableFromJSON(
  (await vg.coordinator().query(sql, {type: 'arrow', cache: false})).toArray()
)

```

Your data tables will now let you select “vg\_data\_arrow” as table and correctly show the data.

I also tried to mask the instance by proxying getPrototypeOf, but unfortunately it seems that the format has changed too much to be backwards compatible to 11.

---

<div class="post-metadata">

### Author: ![mootari](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/mootari/32/581_2.png) [@mootari](https://talk.observablehq.com/u/mootari)
#### Post date: [September 11, 2024, 12:41pm UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/13 "2024-09-11T12:41:27Z")

</div>

> [@mootari](#):
>
> which then fails DuckDB’s instance checks

I was misremembering. It’s actually Arrow that performs the instanceof checks:

> <https://github.com/apache/arrow/blob/27acf8bf6e16d4c53e1ed59d5ef46ac7db0306ea/js/src/ipc/writer.ts#L172>

---

<div class="post-metadata">

### Author: ![cn00](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.observablehq.com/cn00/32/9217_2.png) [@cn00](https://talk.observablehq.com/u/cn00)
#### Post date: [September 12, 2024, 5:12am UTC](https://talk.observablehq.com/t/data-table-cell-arrow-data-support/9737/15 "2024-09-12T05:12:06Z")

</div>

Thank a lot for your time. The performance is not the first level for me at this time . I can use the array instead of arrow. I’m using `vg.coordinator().query()` just for get data from http on AWS s3. I had find the solution you had answer in the other channel, thanks again! 😄
