[data] Include column name and target type in ArrowConversionError

[goutamvenkat-anyscale]

Why are these changes needed?

ArrowConversionError previously only showed the data that failed to convert, making it hard to identify which column caused the issue. For example, the raw Arrow error looks like:

File "pyarrow/array.pxi", line 405, in pyarrow.lib.asarray
File "pyarrow/array.pxi", line 375, in pyarrow.lib.array
File "pyarrow/array.pxi", line 46, in pyarrow.lib._sequence_to_array
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowTypeError: Expected bytes, got a 'numpy.float32' object

This gives no indication of which column or what type conversion was attempted. With this change, the wrapped error now includes the column name and inferred target type:

Before:

Error converting data to Arrow: [b'hello', 2.0]

After:

Error converting column 'my_column' (target type: binary) to Arrow: [b'hello', 2.0]

Repro script

import ray
import numpy as np

ray.init()

ds = ray.data.range(2)

def mix_types(batch):
    return {"my_column": [b"hello", np.float32(2.0)]}

ds.map_batches(mix_types, batch_size=2).write_parquet("/tmp/out")

Related issue number

N/A

Checks

• I've signed off every commit.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've added any new APIs to the API Reference.
• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 93cb773. Configure here.}

[gemini-code-assist]

Code Review

This pull request enhances the ArrowConversionError class to include optional column name and target type information, providing more context in error messages. The review feedback identifies a critical issue where a NameError could occur in _convert_to_pyarrow_native_array if an exception is raised before pa_type is defined. Additionally, there is a suggestion to refactor the error message construction logic for improved readability.

[gemini-code-assist]

The message construction logic can be made more concise and arguably more readable by handling the column_name is None case first, and combining the f-string for the other case.

        if column_name is None:
            message = f"Error converting data to Arrow: {data_str}"
        else:
            type_info = f" (target type: {pa_type})" if pa_type is not None else ""
            message = f"Error converting column '{column_name}'{type_info} to Arrow: {data_str}"

[iamjustinhsu]

when can column_name be present but pa_type None?

[iamjustinhsu]

also, what are the implications if you did this instead?

type_info = f" (target type: {pa_type})"

regardless of pa_type check?

[goutamvenkat-anyscale]

pa_type can be None if it fails before _infer_pyarrow_type.