TOOL | Eval statistics
Find eval rows that mean the same thing in different words
Paste an eval set and pick the text column. An in-browser model groups rows that ask the same question in different words, reports how many collapse, and exports the pairs with a run stamp.
Two rows that ask the same question in different words pass the deterministic checkers clean. Each duplicate counts one item twice: the rate moves in whichever direction the model answers that item, and every interval is narrower than it should be. This page loads an embedding model in your browser, groups the rows whose meaning overlaps, and reports the sample size you are left with. It is the sixth rung of the contamination ladder.
Showing your last valid result. Update the inputs above to recompute.
Groups found
2
2 groups over 6 rows scanned at 0.90. Connected groups at the chosen threshold. Six-row example, on the column "input".
- Rows scanned
- 6
- Rows in groups
- 5
- Duplicate share
- 83.3%
- Effective rows each group collapsed to one representative
- 3
- Closest pair highest cosine in the file, threshold or not
- 0.939
Rows in groups as a share of all scanned rows, to one decimal. Cosine to three decimals.
Collapse each group to one representative and this is the row count the rest of your sizing math should use. A useful rule of thumb: keep the row whose wording is closest to what production users send, or the one with the richest reference answer. The golden-set size planner starts from this number.
2 groups hold 5 of 6 rows scanned, 83.3% of the file. Collapse each group to one row and 3 rows are left. 1 group was joined through a middle row: the lowest internal pair sits under the threshold, so the two ends may not be duplicates of each other.
Every pair at or above the threshold, sorted by cosine descending. Read the raw text of both rows before cutting either one.
| Group | Row A | Row B | Cosine |
|---|---|---|---|
| 1 | q1: What are the steps to reset my password? | q2: How do I reset my password? | 0.939 |
| 1 | q2: How do I reset my password? | q3: I forgot my password. How can I reset it? | 0.930 |
| 2 | q4: Which city serves as the capital of France? | q6: What is the capital city of France? | 0.922 |
The connected groups, each with the lowest cosine across all its internal pairs.
| Group | Rows | Lowest pair | Note |
|---|---|---|---|
| 1 | q1, q2, q3 | 0.872 | Chained: the lowest internal pair is under the threshold, so the ends may not match each other. |
| 2 | q4, q6 | 0.922 |
The groups here carry no labels. Counting the distribution over slices you did name is the balance analyzer.
These figures are a worked example. Replace every cell with your own data.
v = normalize(mean(hidden, mask)); pair when v_i · v_j >= thresholdHow?
How this is calculated
The model is a sentence transformer (all-MiniLM-L6-v2, int8-quantized ONNX, six layers, 384 dimensions) that runs entirely in the browser through WebAssembly. The asset is downloaded once from this site, cached by the browser, and never touches your data. A different architecture is not interchangeable: a plain BERT's CLS vector, compared by cosine, performs worse than averaging static word vectors (Reimers and Gurevych, 2019), because the checkpoint was trained under a Siamese objective that produces sentence-level signal only through mean pooling.
For each row the model reads the text column, tokenizes it with its own WordPiece vocabulary, and runs the encoder. The last hidden state is a matrix of token embeddings. Mean pooling sums the embeddings at positions where the attention mask is 1, divides by the mask count, and ignores padding. The pooled vector is then L2-normalized to unit length.
Similarity between two rows is the dot product of their unit vectors, which is the cosine. The range is [-1, 1]; for this model family, values sit in roughly [0, 1]. The page displays cosine to three decimal places rather than the site's usual one, because the result turns on a 0.01 difference in the threshold and a one-decimal display would show the same digit for both sides of it.
A pair is an edge whenever the cosine meets or exceeds the threshold. The page builds the undirected graph of edges and finds its connected components. A component with two or more members is a group. For each group, the minimum cosine across all internal pairs (including non-edges) is printed. When that minimum is below the threshold, single linkage joined the group through a middle row: A matched B and B matched C, but A and C did not match each other. Marking those groups is cheaper than running a second clustering algorithm and it makes the failure visible rather than silent.
Effective rows equals scanned rows minus the rows that would collapse if each group kept one representative. Rows left equals rows scanned minus rows in groups plus one per group: each group keeps one representative. It is the row count the golden-set size planner should receive.
There is no confidence interval on the duplicate share. The rate is a census: every row was embedded and every pair was compared, so there is no sampling error to bracket. Printing an interval here would imply a sampling step that does not exist.
The token window is 256 word pieces. A longer row is truncated by the tokenizer and compared on its opening only. The page counts those rows and says so beside the result. The pair loop computes n(n-1)/2 dot products of 384 dimensions. At 2,000 rows that is about two million pairs, which runs in seconds. The pair loop is not the reason for the row cap. The model runs on the reader's CPU in single-threaded WebAssembly, and at 50 to 150 short rows per second, 2,000 rows is roughly 15 to 40 seconds of embedding time. The deterministic rungs are the ones that scale to 200,000 rows.
A run stamp printed beside the result and carried in both exports records the model id, asset version, pooling rule, threshold, column and token window. A different model or a different threshold gives different groups, so the stamp is what makes the result traceable rather than reproducible. The rung below this one is reproducible from a printed transform anyone can redo by hand. This page replaces that guarantee with a recorded configuration.
At most 50 pairs and 20 groups render, 8 member labels per group, and a long cell is clipped on screen. Both exports carry every pair, every group, and the full text of every cell.
Formula: v = normalize(mean(hidden, mask)); pair when v_i · v_j >= threshold
Questions
How is this different from the near-duplicate row detector?
The near-duplicate detector compares characters. It normalizes spacing, casing, quotes and Unicode form, then groups the rows whose normalized text is identical. Two rows that ask the same question in entirely different words pass it clean. This page embeds the text and compares meaning, so it catches the paraphrases the character-level check cannot reach.
Why does the model run in my browser?
Your file is processed in this browser tab, and this page never uploads it. The embedding model is downloaded from this site and runs in WebAssembly. No inference request leaves the tab. The trade-off is speed: the model embeds roughly 50 to 150 short rows per second on a current laptop, so the page caps the file at 2,000 rows.
Can I change the threshold after the run?
Yes. The embeddings stay in memory until you clear them or pick a new file. Moving the slider recomputes the edges, the groups and the rates without re-embedding.
What does "chained group" mean?
Single linkage joined rows through a middle row. Row A matched row B and row B matched row C, but A and C sit below the threshold. The group is marked and its minimum internal cosine is printed so you can inspect the ends yourself.
The deterministic checkers handle 200,000 rows. Why does this one stop at 2,000?
The deterministic checkers compare strings and tokens, which a CPU handles in milliseconds per row. This page runs a transformer model on the reader's CPU in single-threaded WebAssembly. At 200,000 rows the embedding alone would run for hours. Run the deterministic rungs first and bring a smaller slice here.
Is the result reproducible?
Not the way the rung below it is. The near-duplicate detector publishes its normalization steps and anyone can redo the check by hand. This page depends on a model, a quantization, a pooling rule and a threshold. Change any of them and the groups change. The run stamp in both exports records the full configuration, so the result is traceable.