How CoralDuck works
Two kinds of thing happen here: you create cohorts, and you characterize them. They are not two phases — what you learn from characterizing a cohort is what tells you how to cut the next one, so the two run as a loop.
The loop
The cohort tree
The upper panel holds every cohort you have made and how you made it. Cohorts and the operations between them are nodes joined by edges, so the tree is the record of the analysis rather than a picture drawn next to it. The first cohort holds every row of the table you loaded and is created for you.
A table loaded from a URL, or one of the demos, is named by the address you are on:
/app/?data= and the URL, or /app/?demo= and the demo. Send
that address and it opens the same table for whoever follows it, their own browser
fetching it from the same host. It carries the table and its missing-value marker, not
your cohorts. A file from your disk has no such address, since it is on your disk alone.
Selecting a cohort gives it a color, and that color is the cohort’s identity everywhere else — in the charts, in both tables, in the comparison matrix.
The overview
Cohorts and the operations between them
The tree is the record of the analysis, not a picture drawn beside it.
Creating cohorts
Selected cohorts drop into the action view below, which is three side-by-side areas: Input, Operation, Output.
The middle area holds the operation. It applies to every input cohort at once, and each input produces its own outputs, lined up row by row with the input they came from.
Context columns are a separate thing, and they live in the Input and Output areas instead: added per side, pinnable, and there to describe the cohorts — so you can spot the selection bias and the distribution drift the operation introduced. One row per cohort and one column per attribute, so many attributes across many cohorts stay readable side by side.
Filter and split
Filter keeps what you selected and makes one new cohort from it, however many attributes you selected across — one filter, one cohort, not a chain of them. Split divides the input into one cohort per selection. Both work off the same chart and the same clicks, and both can also hand you everything you did not select, as a cohort.
Filter
One cohort out
The selection becomes a single new cohort, and the badge previews its size.
Split
One cohort per selection
Each output takes its own row, lined up with the input it came from.
The projection layout
The cohort tree has a second layout. The switch in the Overview’s header, Tree | Projection, draws the same tree over two coordinates of your data: every row of the table is a point, and every cohort is a diamond at the mean of its rows, joined by the tree’s edges. A selected cohort colors its points and draws its contours, rings that show where its rows are dense. Coral had only the tree; this layout follows Eckelt et al. 2023.
You pick the two coordinates in the layout itself: any two numeric columns, such as a t-SNE or UMAP you computed elsewhere, or two plain measurements. No coordinates in your table? Compute embedding runs UMAP in your browser over the attributes you pick. It is seeded, so the same attributes and settings draw the same map every time. The result is two new columns, marked as computed, and they come back when you reload.
The projection layout
Two numeric columns, or a computed embedding
Each selected cohort draws its points in its own color, with its contours.
The lasso and clusters
Drag in the layout to draw a lasso. The rows inside it are marked and counted, and nothing else changes until Filter by lasso opens Filter with it for every selected cohort. So a cohort cut from the map is still checked in the action view and added with “Add to Cohort Tree”, like any other.
Find clusters colors the points by the dense regions of the map, and one knob, from “Fewer clusters” to “More clusters”, sets how many there are. Split by clusters adds the clusters as a column and opens Split on it, so each cluster becomes a cohort.
A lasso
The rows inside, counted and handed to Filter
The rest of the points fade while the lasso is there. Below, Filter draws the same loop over its scatterplot.
Clusters
The dense regions of a computed embedding
Rows in no dense region are gray and counted as unclustered.
Seeing the distributions
View is the way into the data: how the values of one or two attributes fall across the cohorts you have selected. Every cohort is drawn in the same chart, and numeric bins are computed once from the whole table rather than per cohort, so two cohorts are always comparable by eye. Which chart you get follows from how many attributes you picked and what kind they are.
Histogram
One categorical or numeric attribute
One bar per value, grouped by cohort. A wide categorical draws its 24 most common values and collapses the rest into one gray (other) bar, which is a real bar — clickable, filterable, splittable.
Density plot
One numeric attribute
A binned kernel density estimate on a fine grid — exact, not sampled — with the curves of several cohorts drawn over each other and a missing-values line underneath.
Kaplan-Meier
A time column and a status column
You choose which column is the time and which is the event, rather than the tool assuming a schema. Curves for several cohorts are drawn together.
Box plot
One numeric and one categorical
A box per category, per cohort.
Scatterplot
Two numeric attributes
Every cohort in its own color. It caps at 5,000 points per cohort and says “showing N of M” rather than quietly thinning the picture.
Area chart
Two categorical attributes
One area per combination of the two values, per cohort.
Prevalence
Prevalence is the proportion of items with some characteristic within a reference group — the share of a population that your cohort is. Pick the cohort whose characteristic you care about, then build the reference by keeping or skipping each of the filters and splits that made it.
The reference is a predicate rather than a cohort, so it does not have to exist in the tree to be asked about. The interval around the estimate is Agresti-Coull, which stays honest at rates near zero and one where the textbook interval does not, and the correction for rows with no value is applied per predicate.
Prevalence
A cohort against a reference you define
Each input gets its own bar, its own reference size and its own interval.
Compare
Looking at two distributions side by side raises a hypothesis; Compare is how you check it. It lays out the full square of the cohorts you selected, one card per attribute, and runs the appropriate test in every cell — so a pattern you noticed in one chart is tested against every other attribute at the same time.
Significance is painted as ink opacity on one achromatic ramp, never as a hue, because hue already names cohorts. A cell that cannot be tested is a legible dash rather than a blank. Clicking a cell opens a panel that names its own statistic in prose, with the effect size for a numeric pair.
The matrix
Every pair, every attribute
One card per attribute, in the order you selected the cohorts.
One cell, in prose
The test, the score and the effect
Rows with no value are dropped, and the card says how many.
Characterize
Compare tests the attributes you pick. The Characterize operation asks the question the other way round: of all the attributes, which ones tell the selected cohorts apart? It is Coral’s Kokiri operation, ported to run in your browser.
Select two or more cohorts, choose Characterize from the operations, and press Run. A random forest learns to tell the cohorts apart and ranks the attributes by how much each one helped. By default it leaves out the cohorts’ defining attributes, the filters and splits that made them, so the forest does not just find those again.
The results fill in while the forest grows. Attribute importance ranks the attributes, and each one is confirmed, tentative or rejected, judged against a shuffled copy of itself. Cohort differentiation gives the accuracy, where each row is predicted only by trees that did not train on it. Item predictions lists each item’s chance of belonging to each cohort, and the cohort overview maps the items by those predictions, so items the forest finds alike sit together.
Stop keeps the forest grown so far. Exclude selected attributes and Limit to selected attributes run it again on fewer attributes, a step at a time, until Reset.
One finished run
Importance, accuracy and the items
Rejected attributes are hidden behind a Show link, and tentative ones are dimmed.
Inspecting the items
Every cohort is ultimately a set of rows, and Inspect Items is where you read them. Choose the attributes you want as columns and the rows of the selected cohorts appear, with a membership column saying which cohort each row belongs to.
The table is virtualized against the database: a million rows stay a million rows in DuckDB and a screenful of elements in the tab. Sorting and column filters run as queries, so they are as fast on the whole table as on a page of it. A column filter is a lens — it changes what you see, creates no cohort, and is gone on reload.
Two exports leave from here. Download table writes what you are looking at, as sorted and filtered, with one boolean column per cohort. Download rows on a cohort in the tree writes that one cohort with every column. Both carry the source file’s line order, which is the only id an arbitrary CSV has.
The item table
The rows behind the cohorts
Numbers are drawn as bar length against the whole table, in one flat color.
Not here yet
Coral’s onboarding tooltips are not ported yet.
The news entry says what this release is, and the issue tracker is where the rest of it lives.