> For the complete documentation index, see [llms.txt](https://docs.algenta.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.algenta.ai/benchmarks/dataframe-groupby.md).

# Dataframe group-by

Group-by aggregation on the H2O db-benchmark, 0.5 GB tier, through Algenta's composed route — six of six questions ahead of both DuckDB 1.5.5 and Polars 1.43.1 on the minimum.

The H2O db-benchmark groupby suite is the standard public comparison for dataframe group-by performance. This page runs questions q1-q5 and q7 of the 0.5 GB tier — 10,000,000 rows, K=100 — against Polars and DuckDB on the same data, on the same machine, in the same interleaved sessions.

Algenta answers group-by with a composed router that inspects each question's key shape and picks one of three scan strategies (dense, packed or a fused-radix route) — see [route](#results) per question below.

**Ratio convention on this page: baseline time divided by Algenta time. Above 1.00 means Algenta is faster.**

## Results

Minimum, median and maximum ratio across two interleaved sessions of six timed calls per engine per question (three repetitions, two cycles, engine order rotated every cycle) — twelve timed calls per engine per question in total.

| Question                                 | groups  | vs DuckDB (min / median / max) | vs Polars (min / median / max) | route  |
| ---------------------------------------- | ------- | ------------------------------ | ------------------------------ | ------ |
| q1 — sum(v1) by id1                      | 100     | **1.25 / 1.31 / 1.07**         | **1.95 / 2.65 / 6.12**         | dense  |
| q2 — sum(v1) by id1, id2                 | 10,000  | **1.55 / 1.41 / 1.21**         | **7.94 / 8.50 / 11.08**        | packed |
| q3 — sum(v1), mean(v3) by id3            | 100,000 | **2.77 / 3.41 / 2.59**         | **2.66 / 3.64 / 3.49**         | dense  |
| q4 — mean(v1), mean(v2), mean(v3) by id4 | 100     | **1.53 / 1.28 / 1.30**         | **1.97 / 2.40 / 10.74**        | dense  |
| q5 — sum(v1), sum(v2), sum(v3) by id6    | 100,000 | **4.18 / 4.31 / 3.28**         | **3.18 / 2.86 / 5.39**         | dense  |
| q7 — max(v1), min(v2) by id3             | 100,000 | **2.25 / 2.51 / 2.10**         | **2.16 / 1.91 / 2.58**         | dense  |

**Six of six questions, every ratio above 1.00, on the minimum.** That is the rule this benchmark enforces before calling anything a win. q7 is reported as its two components (`max`, `min`) rather than their difference, which the official suite computes; subtracting them afterward is O(groups) extra work and does not change which engine is faster.

## Ratio to the memory-bandwidth ceiling

Ceiling per question is the bytes it must touch, at this engine's float64 column width, divided by 350.9 GB/s (measured STREAM bandwidth on this machine). No engine here is close to it:

| Question | ceiling  | Algenta (min) | x ceiling | DuckDB (min) | x ceiling | Polars (min) | x ceiling |
| -------- | -------- | ------------- | --------- | ------------ | --------- | ------------ | --------- |
| q1       | 0.456 ms | 2.81 ms       | 6.2x      | 3.52 ms      | 7.7x      | 5.48 ms      | 12.0x     |
| q2       | 0.684 ms | 10.26 ms      | 15.0x     | 15.87 ms     | 23.2x     | 81.41 ms     | 119.0x    |
| q3       | 0.684 ms | 21.94 ms      | 32.1x     | 60.79 ms     | 88.9x     | 58.33 ms     | 85.3x     |
| q4       | 0.912 ms | 4.11 ms       | 4.5x      | 6.28 ms      | 6.9x      | 8.11 ms      | 8.9x      |
| q5       | 0.912 ms | 18.70 ms      | 20.5x     | 78.18 ms     | 85.7x     | 59.48 ms     | 65.2x     |
| q7       | 0.684 ms | 25.53 ms      | 37.3x     | 57.34 ms     | 83.8x     | 55.15 ms     | 80.6x     |

Algenta is closer to the ceiling than either reference on every question, and furthest from it — 32-37x — on the two 100,000-group questions with more than one aggregate column, where the serial merge across the group table is the largest remaining cost.

## Method

* **Dataset.** The H2O db-benchmark G1 groupby data at the 0.5 GB tier: 10,000,000 rows, K=100, generated by a seeded, deterministic generator following the official datagen spec. Group cardinalities are 100 (`id1`, `id4`), 10,000 (`id1` x `id2`) and 100,000 (`id3`, `id6`). The benchmark's categorical columns are emitted as integer dictionary codes for every engine equally — every group key measured on this page is **int64, not a string column**. That distinction matters for Polars specifically: a separate measurement on this project found Polars paying a 2.15x cost increase on this same query shape when the group key is a string column instead of a fixed-width integer one. No engine on this page is carrying that tax.
* **Timing.** Minimum, median and maximum across two interleaved sessions, three repetitions x two cycles each, all three engines back to back with the engine order rotated every cycle so no engine is always measured first or last. Every engine aggregates in-memory columns; no CSV parse is timed.
* **Versions.** Polars 1.43.1 and DuckDB 1.5.5, both at their default parallelism, asserted at runtime before any timing.
* **Hardware.** Apple M4 Max, 16 physical cores, 128 GiB, Mojo 0.26.3.0.dev2026042305 compiled to a binary — the JIT is not the artifact these numbers describe.
* **Correctness.** Every group key, value, presence flag and row count is compared **on the bits** between Algenta's composed answer and the shipped, unrouted engine on the full 10,000,000-row result before any timing is trusted. Cross-engine, a checksum — the sum over groups of every aggregated column — agrees exactly on q1, q2 and q7, and to a relative difference no worse than 1.2e-14 on q3, q4 and q5 (floating-point summation order, not a defect).
* **Semantics.** Group-by results preserve first-appearance order. NaN values count and contribute to no extremum: a NaN row is counted in its group, and a NaN never becomes a group's min or max.

## Scope

* **One tier measured.** These results are the 0.5 GB tier (10,000,000 rows). The 5 GB and 50 GB tiers of the suite are not yet measured.
* **q6, q8 and q9 are out of scope.** Grouped median (q6), largest two per group (q8) and grouped correlation (q9) require grouped order statistics and grouped co-moments, which the query plane does not provide. They are omitted rather than approximated.
* **Not comparable to the public leaderboard.** The official db-benchmark groups by string keys; here the categorical columns are integer dictionary codes for every engine equally, which is also why the dataset note above states it plainly rather than leaving it implicit. These timings compare the three engines against each other on this data and do not reproduce published leaderboard entries.
* **This is one operation, not a query planner.** DuckDB and Polars both do far more than group-by; this page says nothing about joins, windows, spilling to disk, or a dataset larger than memory.
* **Timings are machine-specific and were taken on a contended host.** The absolute milliseconds hold for the hardware named in [Method](#method) at 1-minute load 46-67, never idle; the ratios, taken with all three engines interleaved under that same load, are the result.

## Reproduce

```
mojo build -I mojo/engine benchmarks/mojo_benchmarks/bench_h2o_composed.mojo -o /tmp/bench_h2o_composed
python benchmarks/h2o_groupby/generate.py
```

```
mojo run benchmarks/mojo_benchmarks/bench_h2o_composed.mojo q1 6 0
```

```
python benchmarks/h2o_groupby/run_composed_interleaved.py --binary /tmp/bench_h2o_composed
```

The interleaved driver asserts the Polars and DuckDB versions at runtime, refuses to run against the wrong ones, and aborts rather than prints a number if any engine's result differs from the others on the bits. Pass `--reps 6 --cycles 2` or higher to widen the sample on any one contested question.

## See also

* [Group data locally with Query.group\_by()](/guides/local-query-groupby.md) — how to call this same composed engine on your own data.
* [LLM tool-calling uplift](/benchmarks/llm-tool-calling-uplift.md) — an LLM using a group-by tool over this same kind of data.
* [Benchmarks](/benchmarks/overview.md) — methodology and what is not published.
* [Compute kernels](/benchmarks/compute-kernels.md) — the matrix kernel results.
* [Runtime library catalog](/engine-runtime/runtime-library-catalog.md) — every module and function the runtime exposes.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.algenta.ai/benchmarks/dataframe-groupby.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.