SmartQueryTools

Extract Unique Values from Parquet Files Online

Extract all distinct values from any column in Parquet files directly in your browser. Download the unique values list — no upload required.

How to extract Unique Values from Parquet files

  1. Drop your file onto the upload area. The row count is shown and the first column is pre-selected.
  2. Choose the column you want to list in the Column dropdown.
  3. Click Extract Unique Values. The distinct values appear in a single column called value, sorted in ascending order.
  4. Click Download to save the list in the same format you uploaded, with _unique added to the file name.

Your file is processed locally in your browser and is never uploaded. The free limit is 50 MB per file; larger files work if your device has the memory for them.

Worked example

An HR team is building a department filter for a careers page from a job postings export. They need the list of departments that actually appear in the data.

Input (Parquet)

Parquet file (binary, columnar) — shown as a table with its schema

job_idtitledepartmentremote
J-310Backend EngineerEngineeringtrue
J-311Account ExecutiveSalesfalse
J-312QA Analystengineeringtrue
J-313Payroll SpecialistFinancefalse
J-314Office CoordinatorNULLfalse
J-315Sales ManagerSalesfalse

Schema: job_id VARCHAR, title VARCHAR, department VARCHAR, remote BOOLEAN

Settings

  • Column: department

Result

value
Engineering
Finance
Sales
engineering

Six rows give four distinct values. Sales appears twice but is listed once. The null department is left out. Engineering and engineering are different values because matching is case-sensitive. The sort puts all capitalised words before lowercase ones, which is why engineering comes last. The list reveals a data entry problem to fix before it goes on the site.

Working with Parquet files

The output is a one-column Parquet file whose value column has the same type as the source. A list of distinct dates stays a DATE column and a list of product IDs stays an integer. That makes it ready to use as a dimension or lookup table in the same pipeline.

Distinct is exact and uses the stored values, not a display format. Timestamps are compared to the microsecond, so a timestamp column rarely has fewer unique values than rows. Truncate it to a date first if you want the list of days. Parquet writers often dictionary-encode low-cardinality columns. This tool does not read that dictionary. It scans the column, so the list reflects the values actually used in rows. A distinct list is also a quick way to check a categorical column against an allowed set before loading, for example that a country column holds only ISO codes.

Frequently Asked Questions

Does the Parquet output keep the column type?

Yes. The single value column has the same type as the column you chose, so dates stay dates and integers stay integers.

Can I get unique values from a nested Parquet field?

You can choose a struct column, and each distinct whole struct is listed. For a single nested field, select it as its own column in the SQL Query tool first.

Are null values included in the list?

No. Nulls are excluded. Empty CSV fields and blank Excel cells load as nulls, so they are left out too. An empty string in a JSON or Parquet file is a real value and is listed. Use Count By if you need to know how many rows are null.

Is the matching case-sensitive?

Yes. "Sales" and "sales" are listed separately, and capitalised values sort before lowercase ones. Run Convert Case first if you want them merged.

Is there a limit on the number of unique values?

The on-screen table shows the first 500 values. The heading gives the full count, and the downloaded file contains all of them, limited only by your browser's memory.

Related Tools