Detect Outliers in Parquet Files Online
Detect statistical outliers in Parquet files directly in your browser. Flag or remove rows where numeric values exceed a chosen number of standard deviations from the mean — no upload required.
How to detect Outliers in Parquet files
- Drop your file onto the upload area. It is loaded into the in-browser engine and the first 200 rows are shown.
- Pick a threshold in standard deviations from the mean: 1.5σ, 2σ (the default), 2.5σ or 3σ.
- Pick an action: Flag outliers adds an is_outlier column, Keep outliers returns only the outlier rows, and Remove outliers returns the rest.
- Choose the columns to check. Every numeric column is selected to start with. Click a column to toggle it, or use All and None. A row counts as an outlier if any selected column is beyond the threshold.
- Click Detect Outliers. The count shows how many rows were flagged, kept or left. Review the result, then download it in the same format you uploaded.
Your file is processed locally in your browser and is never uploaded. The free limit is 50 MB per file; larger files work if your device has the memory for them.
Worked example
A food distributor logs one temperature reading from each of six freezer units. One unit has a failing compressor and reads far warmer than the rest.
Input (Parquet)
Parquet file (binary, columnar) — shown as a table with its schema
| unit | location | temp_c |
|---|---|---|
| FZ-01 | Bay 1 | -18.2 |
| FZ-02 | Bay 1 | -18.6 |
| FZ-03 | Bay 2 | -17.9 |
| FZ-04 | Bay 2 | -18.4 |
| FZ-05 | Bay 3 | -9.5 |
| FZ-06 | Bay 3 | -18.1 |
Schema: unit VARCHAR, location VARCHAR, temp_c DOUBLE
Settings
- Threshold: 2σ
- Action: Flag outliers
- Columns to check: temp_c (the only numeric column)
Result
| unit | location | temp_c | is_outlier |
|---|---|---|---|
| FZ-01 | Bay 1 | -18.2 | false |
| FZ-02 | Bay 1 | -18.6 | false |
| FZ-03 | Bay 2 | -17.9 | false |
| FZ-04 | Bay 2 | -18.4 | false |
| FZ-05 | Bay 3 | -9.5 | true |
| FZ-06 | Bay 3 | -18.1 | false |
The mean is −16.78 °C and the sample standard deviation is 3.58 (both rounded), so 2σ is 7.15 degrees. FZ-05 is 7.28 above the mean, a z-score of 2.04, so it is flagged. The faulty reading also inflates the mean and the deviation it is judged against. At 2.5σ nothing would be flagged in a file this small.
Working with Parquet files
Parquet files for anomaly checks are often wide, with many metric columns. With every numeric column selected, a row is flagged if any one of them is extreme. The more columns you check, the more rows get flagged. Click None, then select the two or three metrics you care about. INT, FLOAT, DOUBLE and DECIMAL columns are all eligible. Timestamps and nested struct fields are not.
The mean and standard deviation for each column are computed over the whole file in one pass, not per row group or per partition. A file mixing very different populations, such as several sensors, is judged against one combined average. Filter it to one population first if that matters. In Flag mode, is_outlier is added as a BOOLEAN at the end of the schema. In Keep and Remove mode the schema is unchanged and only the row count differs.
Frequently Asked Questions
Can I detect outliers per group in a Parquet file, for example per sensor?
Not in one run. The mean and deviation are computed over the whole file. Filter the file to one group at a time, or use the SQL Query tool with PARTITION BY in the window functions.
What type is the is_outlier column in the Parquet output?
BOOLEAN. It is always true or false. A null value counts as false.
Which method does the tool use?
A z-score test. A value is an outlier when it is more than the chosen number of sample standard deviations away from its column's mean. The mean and deviation come from the whole file. IQR and median-based methods are not offered.
Why are no outliers found at 3σ in my small file?
With n rows, no value can be more than (n − 1) / √n sample standard deviations from the mean. That means 2σ needs at least 6 rows, 2.5σ needs 9, and 3σ needs 11 before anything can be flagged at all. Extreme values also inflate the deviation they are measured against.
How are blank values handled?
They are ignored when the mean and deviation are computed, and a blank is never an outlier. In Flag mode the row gets false unless another column flags it. Remove outliers keeps the row, and Keep outliers leaves it out.
Related Tools
Filter Parquet Files Online
Filter rows in Parquet files by column value, directly in your browser. Your data stays on your device.
Aggregate Parquet Files Online
Group and aggregate Parquet files by any column directly in your browser. Calculate sum, average, min, max, and count for any numeric column — no upload required.
Fill Empty Values in Parquet Files Online
Fill empty and null values in Parquet files with a custom replacement value, directly in your browser.
Detect Outliers in CSV Files Online
Detect statistical outliers in CSV files directly in your browser. Flag or remove rows where numeric values exceed a chosen number of standard deviations from the mean — no upload required.
Detect Outliers in Excel Files Online
Detect statistical outliers in Excel files directly in your browser. Flag or remove rows where numeric values exceed a chosen number of standard deviations from the mean — no upload required.
Detect Outliers in JSON Files Online
Detect statistical outliers in JSON files directly in your browser. Flag or remove rows where numeric values exceed a chosen number of standard deviations from the mean — no upload required.
Parquet Viewer Online
View and inspect Parquet files directly in your browser. Browse rows, check column names and data types — no upload required, your data stays on your device.
Convert Parquet to CSV Online
Convert Parquet files to CSV format directly in your browser. No upload required — your data never leaves your device.