[Data] Remove column renaming from the read stage

[goutamvenkat-anyscale]

Summary

This PR removes column renaming from the read stage. Reads now only receive the columns they need to scan, while column renames stay as AliasExprs in a Project above the read.

That gives us one clear rule:

• The read stage handles physical column pruning.
• The Project stage handles renames and computed expressions.
• Pure pruning projections can still disappear once pushed into the scan.

Plan Shape

flowchart LR
    subgraph Before
        B1[Project / Filter]
        B2[ReadFiles with column_renames]
        B1 --> B2
    end

    subgraph After
        A1[Project with AliasExpr renames]
        A2[ReadFiles with pruned columns]
        A1 --> A2
    end

Loading

Keeping the Project above the read does not add a separate runtime pass. Physical operator fusion still combines the read and projection into one map operator, so renames run inline with the file read task.

What Changed

• Removed column_renames plumbing from read operators and datasource projection pushdown.
• Simplified projection pushdown so scans receive only a pure column-pruning map.
• Simplified predicate pushdown by removing rebinding between renamed and original column namespaces.
• Kept rename AliasExprs in a Project above reads, preserving the logical output schema.
• Fixed star expansion so only true rename aliases are placed back into the source column position; non-rename aliases like with_column("x", col("a")) are preserved.

Tests

• Updated predicate/projection pushdown tests for the new plan shape.
• Updated Iceberg and Parquet coverage for rename, select, and filter combinations.
• Added regression coverage for with_column("x", col("a")).rename_columns({"a": "b"}) so both x and b are preserved.

[gemini-code-assist]

Code Review

This pull request refactors the projection and predicate pushdown logic to align with DataFusion's architecture, ensuring that column renaming is handled by a Project operator rather than the read stage. This change simplifies the pushdown process by maintaining a single column namespace at the scanner level. Key changes include removing column renaming logic from ReadFiles, IcebergDatasource, and ParquetDatasource, and updating the logical optimizer rules to keep Project operators for renames. Feedback identifies a potential AttributeError when a StarExpr is present during column pruning and notes that the is_pure_prune logic may incorrectly discard Project operators that perform column duplication.

[gemini-code-assist]

The is_pure_prune logic doesn't account for duplicate column references (e.g., ds.select("a", "a")) or a StarExpr combined with other expressions. In these cases, the Project operator is not redundant because it performs duplication that the read stage's column pruning cannot replicate. Discarding the Project would result in an incorrect output schema.

Consider adding checks to ensure that the number of expressions matches the number of unique output column names and that a StarExpr is only considered a pure prune if it's the sole expression.

[ayushk7102]

Nit: can we break this compound conditional up into two booleans? It's a little hard to parse what this means.

i.e.

Suggested change

	is_pure_prune = not any(
	is_pure_prune = no_renames_or_computed and only_col_exprs

[ayushk7102]

LGTM

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit b48a2b3. Configure here.}