# Dynamic rollup object for Arquero
**URL:** https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721
**Category:** Help
**Created:** [March 2, 2021, 3:36am UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721 "2021-03-02T03:36:54Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@MarioDelgadoSr](https://talk.observablehq.com/u/MarioDelgadoSr)
#### Post date: [March 2, 2021, 3:36am UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/1 "2021-03-02T03:36:54Z")
Is there a JS syntax that can be used in the cell _ **rollupObj** _ in this [notebook](https://observablehq.com/@mariodelgadosr/dow-jones-industrial-average-correlation-analysis-with-ar) that does not utilize the _ **eval** _ method?
```auto
rollupObj = {
const rollupObj ={};
correlationPairings
.forEach(({metric_1, metric_2}) => {
rollupObj[`${metric_1}_${metric_2}`] =
eval(`d => op.corr(d["${metric_1}"], d["${metric_2}"])`);
// d => op.corr(d[metric_1], d[metric_2]) // this does not work
});
return rollupObj;
}
```
---
### Author:  [@severo](https://talk.observablehq.com/u/severo)
#### Post date: [March 2, 2021, 9:58am UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/2 "2021-03-02T09:58:24Z")
Maybe related to [Get month name in Arquero table - #2 by severo](https://talk.observablehq.com/t/get-month-name-in-arquero-table/4432/2)
---
### Author:  [@MarioDelgadoSr](https://talk.observablehq.com/u/MarioDelgadoSr)
#### Post date: [March 2, 2021, 2:21pm UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/3 "2021-03-02T14:21:37Z")
Your references ([1](https://talk.observablehq.com/t/get-month-name-in-arquero-table/4432/2),[2](https://observablehq.com/@bmschmidt/exploring-changing-us-college-majors-with-arquero#comment-858c591ca4faecc0)) were extremely helpful in understanding what exactly is going on; Thank You!
The _ **’A few words about table expressions and op…'** _ section in [Introducing Arquero](https://observablehq.com/@uwdata/introducing-arquero) really explains the issue.
Unfortunately, for my use case, I’m dynamically creating new columns/attributes for the rollup-table and **can’t reference columns/attributes that are being generated in the rollup** as parameters for the [op.corr method](https://uwdata.github.io/arquero/api/op#corr).
The [params](https://uwdata.github.io/arquero/api/table#params) method/pattern is only applicable for calculating values as a function of the current table row being processed. It is a similar scenario to what @bmschmidt explained in [this notebook](https://observablehq.com/@bmschmidt/exploring-changing-us-college-majors-with-arquero#comment-858c591ca4faecc0).
So my use of the [eval](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval) appears to be the only option so far for my specific situation.
---
### Author:  [@bmschmidt](https://talk.observablehq.com/u/bmschmidt)
#### Post date: [March 3, 2021, 11:57am UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/4 "2021-03-03T11:57:31Z")
I think the reason this is difficult is that the data columns are not tidy. If you fold the data into a long format first, the cross-product correlation is relatively straightforward in arquero: the only weird bit is a custom join function to avoid duplicating keys on the left and right.
```auto
const long = aq
.from(data)
.fold(aq.not("Date"), {"as": ["company", "price"]})
return long
.join(long, (a, b) => op.equal(a.Date, b.Date) && a.company < b.company)
.groupby("company_1", "company_2")
.rollup(
{correlation: op.corr("price_1", "price_2")})
.orderby(aq.desc("correlation"))
.view()
```
---
### Author:  [@MarioDelgadoSr](https://talk.observablehq.com/u/MarioDelgadoSr)
#### Post date: [March 4, 2021, 4:21pm UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/5 "2021-03-04T16:21:25Z")
> [@bmschmidt](#):
>
> ```auto
> const long = aq
> .from(data)
> .fold(aq.not("Date"), {"as": ["company", "price"]})
> return long
> .join(long, (a, b) => op.equal(a.Date, b.Date) && a.company < b.company)
> .groupby("company_1", "company_2")
> .rollup(
> {correlation: op.corr("price_1", "price_2")})
> .orderby(aq.desc("correlation"))
> .view()
>
> ```
Ben:
Thanks for your elegant approach, but it appears to only apply for the use case where one knows ahead of time exactly which data columns are going to participate in the correlation and what uniquely identifies (keys (verb)) each row…in this case ‘Date’.
Knowing that ‘Date’ is the row identifier allows your logic to negate/remove it from the fold and also use it in the custom join.
The generalized approach is data-driven and dimension-independent. It uses the data profile to determine which columns are measures and then dynamically proceeds with the correlation analyses.
If the data has anything that is being ‘measured’ it will correlate against generic metric\_1, metric\_2 pairings and then rank those correlations. The stock data was used a convenient data source to illustrate that analysis pattern.
I added a ‘Usage’ section to the notebook.
Here’s how you would use _correlationAnalysis_ against the ‘beers’ data from the [Arquero introduction](https://observablehq.com/@uwdata/introducing-arquero):
- import {beers} from “@uwdata/introducing-arquero”
- beersData = beers.objects();
- import {correlationAnalysis} with {beersData as data} from “@mariodelgadosr/dow-jones-industrial-average-correlation-analysis-with-ar”
- correlationAnalysis.view(correlationAnalysis.numRows())
---
### Author:  [@MarioDelgadoSr](https://talk.observablehq.com/u/MarioDelgadoSr)
#### Post date: [March 7, 2021, 5:36pm UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/6 "2021-03-07T17:36:46Z")
A very careful reading of the [API documentation for Table Expressions](https://uwdata.github.io/arquero/api/expressions) results in the following solution:
> …At first glance table expressions look like normal JavaScript functions… _ **but hold on!** _ Under the hood, Arquero takes a set of function definitions, maps them to strings, then parses, rewrites, and compiles them to efficiently manage data internally…
With this is mind, the section of [_ **Limitations** _](https://uwdata.github.io/arquero/api/expressions#limitations) has this critical recommendation:
> …Alternatively, for programmatic generation of table expressions one can fallback to _ **generating a string – rather than a proper function definition – and use that instead** _…
The string _ **inside** _ the eval method can be passed directly as a Table expression. See the solution implemented in cell _**[rollupObj](https://observablehq.com/@mariodelgadosr/dow-jones-industrial-average-correlation-analysis-with-ar#rollupObj)**_.
Be careful to follow the _ **exact** _ string formatting requirements!:
> …_ **using an identifier other than `d` will fail.** _ In contrast, with an explicit function definition you are free to rename the argument as you see fit…
---
### Author:  [@MarioDelgadoSr](https://talk.observablehq.com/u/MarioDelgadoSr)
#### Post date: [March 10, 2021, 2:32pm UTC](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/7 "2021-03-10T14:32:36Z")
The suggestion made by @bmschmidt in [this post](https://talk.observablehq.com/t/dynamic-rollup-object-for-arquero/4721/4) was generalized and compared to the original pattern in this notebook: [Arquero Correlation Speed Test / Mario Delgado / Observable](https://observablehq.com/@mariodelgadosr/arquero-correlation-speed-test).
The difference in time for both analyses is significant; in favor of the dynamic rollup pattern originally introduced in [Dow Jones Industrial Average Correlation Analysis with Arquero / Mario Delgado / Observable](https://observablehq.com/@mariodelgadosr/dow-jones-industrial-average-correlation-analysis-with-ar).