SmartQueryTools

Find Fuzzy Duplicates in YAML Files Online

Find near-duplicate rows in YAML files using Levenshtein edit distance or Jaro-Winkler similarity — all in your browser. Set your own threshold and download the matched pairs as CSV — no upload required.

About this tool

Find and remove near-duplicate rows in a YAML file using string similarity matching — catching typos, abbreviations, and inconsistent formatting that exact deduplication misses. Set a similarity threshold, choose the column to match on, and download a deduplicated file with flagged pairs. Ideal for cleaning customer names, addresses, and product descriptions. All processing runs locally in your browser.

Frequently Asked Questions

How is fuzzy deduplication different from exact deduplication for YAML files?

Exact deduplication removes rows where all columns match perfectly. Fuzzy deduplication finds rows that are similar but not identical — for example "John Smith" and "Jon Smith" or "123 Main St" and "123 Main Street". It uses string similarity to identify near-matches that represent the same real-world entity.

What similarity threshold should I use when fuzzy-deduplicating a YAML file?

Start with 0.85 (85% similarity). A lower threshold catches more duplicates but risks merging distinct records. A higher threshold is more conservative and safer when small differences are meaningful. Review the flagged pairs at your chosen threshold before committing.

Can I fuzzy-deduplicate on multiple columns in a YAML file?

Choose the single column that best identifies the entity — typically a name, address, or description. For multi-column matching such as first name + last name together, use the Concatenate Columns tool to combine them into one column before running fuzzy deduplication.

Is my data private?

Yes — completely. Your file is never uploaded to any server. Everything runs locally in your browser using WebAssembly — processing happens entirely inside your tab. Once you close the tab, nothing is retained.

What is the maximum file size?

The free limit is 50 MB. For larger files, performance depends on your device's available memory — most modern machines handle 500 MB to 1 GB comfortably.

Related Tools