Deduplicate Arrow Files Online

How does the Arrow deduplication tool decide which rows are duplicates?

It compares every column value in each row. Two rows are duplicates only if all columns match exactly. The first occurrence is kept and subsequent duplicates are removed.

Can I deduplicate a Arrow file on specific columns only?

This tool removes rows that are identical across all columns. For column-specific deduplication — for example keeping one row per customer ID regardless of other columns — use the SQL Query tool and write SELECT DISTINCT col1, col2 FROM your_table.

Will deduplication change the order of rows in my Arrow file?

Row order is preserved. The first occurrence of each unique row stays in its original position; later duplicates are dropped.

Is my data private?

Yes — completely. Your file is never uploaded to any server. Everything runs locally in your browser using WebAssembly — processing happens entirely inside your tab. Once you close the tab, nothing is retained.

What is the maximum file size?

The free limit is 50 MB. For larger files, performance depends on your device's available memory — most modern machines handle 500 MB to 1 GB comfortably.