[Data] Fix Dataset.summary() crash on boolean columns

[lonexreb]

Why are these changes needed?

Dataset.summary() crashes with ArrowNotImplementedError on any boolean column:

import ray
ds = ray.data.from_items([{"b": True}, {"b": False}, {"b": True}])
ds.summary()
# pyarrow.lib.ArrowNotImplementedError:
#   Function 'subtract' has no kernel matching input types (bool, double)

DataType.bool() was mapped to _numerical_aggregators, but Std, ApproximateQuantile, and ZeroPercentage rely on PyArrow kernels with no boolean implementation. I probed each default aggregator individually against a bool column on master: Count, Mean, Min, Max, MissingValuePercentage, and ApproximateTopK all work; Std and ZeroPercentage raise ArrowNotImplementedError (and a boolean quantile is not meaningful).

This PR gives boolean columns a dedicated _boolean_aggregators set — count, mean (= fraction of true values), min, max, missing-value percentage, and approximate top-k with k=2 (true/false counts) — and routes boolean dtypes to it in the heuristic fallback path as well.

Related issue number

Fixes #62235

Checks

• I've signed off every commit (DCO).
• Formatting checked with the repo-pinned black==22.10.0.
• Testing strategy:
  • Added test_summary_boolean_column (end-to-end regression: summary() on a bool column with nulls, asserting count/mean/min/max/missing_pct values).
  • Added test_boolean_aggregators unit test; updated the two existing expectations that encoded "boolean is numerical".
  • pytest python/ray/data/tests/test_dataset_stats.py → 39 passed (run locally against master via Ray nightly wheel + setup-dev.py symlinks).

Notes for reviewers / AI-assistance disclosure

• This PR was prepared with AI assistance; I have reviewed every changed line and run the tests above locally.
• Duplicate-work check: no open PR references [data] ds.summary() crashes on boolean columns with ArrowNotImplementedError #62235. @jiangxt2 commented intent to work on this on Jul 2 with the same root-cause diagnosis, but no PR has appeared in the ~7 weeks since; happy to defer or hand off if one is in progress.

[gemini-code-assist]

Code Review

This pull request introduces specialized aggregators for boolean columns in Ray Data stats, resolving an issue where boolean columns would crash the summary() method due to unsupported PyArrow kernels (like subtraction for standard deviation). It maps boolean data types to _boolean_aggregators instead of _numerical_aggregators and adds comprehensive unit tests to verify this behavior. I have no feedback to provide as there are no review comments to evaluate.

[github-actions]

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[lonexreb]

Not stale — rebased onto current master and re-ran the test suite locally (test_dataset_stats.py: 39 passed). Ready for review whenever a maintainer has bandwidth.

[iamjustinhsu]

Hi @lonexreb, how hard would it be to support booleans in those preprocessors? I think I would prefer that over skipping (since all booleans can be converted to numbers)

[lonexreb]

Great suggestion @iamjustinhsu — done in e77b374. Booleans now get the full numerical statistics (treated as 0/1) instead of a reduced set:

• ArrowBlockColumnAccessor.sum_of_squared_diffs_from_mean casts boolean columns to float64 (PyArrow has no boolean subtract kernel) → Std works.
• ZeroPercentage casts boolean columns to int8 before the zero comparison in both its per-block path and the vectorized zero_pct_spec path (equal(bool, int) has no kernel) → zero_pct = fraction of False.
• ApproximateQuantile already handled booleans (datasketches treats them as 0/1).
• DataType.bool() maps back to _numerical_aggregators; the dedicated boolean aggregator set is gone.

summary() on a bool column now reports count, mean (fraction true), min, max, std, median, missing_pct, and zero_pct. The regression test asserts the actual values (e.g. std([1,0,1]), zero_pct == 33.3%), and the casts benefit standalone ds.aggregate(Std(on=bool_col)) / ZeroPercentage(on=bool_col) too.

Rebased onto current master; test_dataset_stats.py passes locally (38 tests).

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit e77b374. Configure here.}

[cursor]

Boolean cast assumes Arrow container

Medium Severity

The new boolean handling reads .type from _to_arrow_compatible_container(), but the pandas accessor returns a Python list. ZeroPercentage on a pandas boolean column then fails with AttributeError instead of treating False as zero.

 

^{Reviewed by Cursor Bugbot for commit e77b374. Configure here.}