Engineering case study 01 · Duplicate detection

100 files, 10 contents: testing exact duplicate detection.

A controlled fixture checks whether Klyr can identify every extra exact copy while refusing to equate same-size files with different contents.

100synthetic regular files
10 × 10unique contents × copies
90extra copies correctly emitted

The question

A duplicate finder should not flag files simply because their names or sizes match. We wanted one controlled test to answer two things: does the scanner identify every additional byte-identical copy, and does it preserve one canonical member in each group?

The fixture

The performance fixture creates ten distinct byte patterns. Each pattern is written to ten separate 2,000-byte files, producing 100 files total. Every group therefore contains one canonical file and nine additional copies.

Input propertyValueReason
Unique contents10Confirms separate hash groups.
Copies per content10Creates nine extras in every group.
File size2,000 bytesExceeds Klyr’s default 1 KB threshold.
Expected result90 extra copies10 groups × 9 non-canonical members.

The two-pass method

Klyr first enumerates regular files and groups them by byte size. Files without a size collision stop there; they cannot be exact duplicates. Candidates in same-size groups are then hashed with SHA-256. Only matching hashes enter the same duplicate group.

This reduces unnecessary reads without weakening the identity rule. Size is a filter, not proof. Content verification supplies the evidence.

Observed result

The test emitted exactly 90 extra copies and passed. The local XCTest performance run repeated the scan ten times and reported an average wall-clock time of 0.011 seconds. Individual samples ranged from approximately 0.008 to 0.028 seconds.

Separate correctness tests also passed for nested duplicates, minimum-size filtering, unique files with different sizes, and—critically—same-size files containing different bytes.

What this result does not prove

This was a synthetic debug-build fixture on the current Apple-silicon development machine, not a benchmark of a full user disk. The reported timing had 51.225% relative standard deviation, largely because the workload is tiny. Real performance depends on file count, file size, storage speed, cloud availability and permissions. The meaningful result is correctness for the disclosed fixture, not a universal “scan in milliseconds” promise.

Product decision

Klyr keeps the two-pass design and describes results as exact duplicates. It does not claim that similar photos, matching names, or same-size documents are duplicates. It also does not automatically decide which path is expendable; users review every group and select what, if anything, moves to Trash.

Read the broader duplicate file finder methodology.