SmartQueryTools

Bin Column in Parquet Files Online

Bucket a numeric column in Parquet files into labelled ranges — equal-width bins or custom edges. Runs entirely in your browser.

How to bin Column in Parquet files

  1. Drop your file. The first numeric column is preselected, and the new column name defaults to its name plus _bin.
  2. Pick the column to bin. Only numeric columns are listed.
  3. Choose Equal-width bins and set how many (2 to 50, default 5), or choose Custom edges and type the break points separated by commas, without thousands separators.
  4. Choose Add as new column or Replace column, then click Bin column.
  5. Check the labels in the preview and download the file in the same format.

Your file is processed locally in your browser and is never uploaded. The free limit is 50 MB per file; larger files work if your device has the memory for them.

Worked example

A food delivery service wants to report how many orders arrived within 30 minutes, within an hour, within two hours, and later than that.

Input (Parquet)

Parquet file (binary, columnar) — shown as a table with its schema

order_refzonedelivery_minutes
D-8812Northside22
D-8813Harbour45
D-8814Northside60
D-8815Airport135
D-8816Harbour30

Schema: order_ref VARCHAR, zone VARCHAR, delivery_minutes BIGINT

Settings

  • Column to bin: delivery_minutes
  • Binning mode: Custom edges
  • Edge values: 0, 30, 60, 120
  • New column name: delivery_minutes_bin
  • Output: Add as new column

Result

order_refzonedelivery_minutesdelivery_minutes_bin
D-8812Northside22< 30
D-8813Harbour4530 – 60
D-8814Northside6060 – 120
D-8815Airport135≥ 120
D-8816Harbour3030 – 60

Each range includes its lower edge and excludes its upper edge, so 30 falls in "30 – 60" and 60 in "60 – 120". Everything below the second edge is labelled "< 30", and the first edge, 0, does not appear in any label. Values at or above the last edge share "≥ 120". The labels are text, ready for Count by value.

Working with Parquet files

Every Parquet integer, float and decimal column is listed, so prices stored as DECIMAL and counts stored as INT32 can be binned directly. Equal-width edges are rounded to six significant digits, which keeps labels short but moves the edges slightly on large ranges. A column running up to 1,234,567,890 gets a top edge of 1234570000, so its maximum falls in the last range rather than under the ≥ label. When the rounding goes down instead, a few of the largest values can move into the ≥ label.

The label column is written as a Parquet string column. With Add as new column the numeric column is kept, which keeps the file useful for later analysis. Replace column removes the numeric column and puts the label at the end of the schema, not where the number was. Jobs that read the file by column position, or expect that column to be numeric, will need updating.

Frequently Asked Questions

Can I bin a DECIMAL column in a Parquet file?

Yes. DECIMAL columns are listed with the other numeric types, and each value is compared against the edges as a number.

Does Replace column keep the column position in Parquet?

No. The numeric column is dropped and the label column is added as the last column. Use Add as new column if position matters.

Why does equal-width binning give me one more label than the number of bins?

The last edge is the column maximum, and values at or above the last edge get their own "≥" label. With 5 bins on values from 0 to 100 the labels are < 20, 20 – 40, 40 – 60, 60 – 80, 80 – 100 and ≥ 100, and only the maximum lands in the last one. For exactly N groups, use custom edges and put the last edge above the maximum.

What happens to empty values?

They stay empty. A NULL number gets a NULL label, so it is not counted in any range. Use Fill Nulls first if you want them in a range of their own.

How do I sort the bin labels in the right order?

The labels are text, so a plain sort puts "100 – 200" before "30 – 60". Sort by the original numeric column instead. Choosing Add as new column keeps that column in the file.

Related Tools