Skip to content

[Data] Fix autoscaler to request previous allocation instead of empty resources when not scaling up#60321

Merged
bveeramani merged 9 commits into
ray-project:masterfrom
machichima:autoscaler-request-previous-allocation
Jan 23, 2026
Merged

[Data] Fix autoscaler to request previous allocation instead of empty resources when not scaling up#60321
bveeramani merged 9 commits into
ray-project:masterfrom
machichima:autoscaler-request-previous-allocation

Conversation

@machichima

@machichima machichima commented Jan 20, 2026

Copy link
Copy Markdown
Contributor

Description

When the DefaultAutoscalerV2 decides not to scale up (e.g., utilization is low), it currently sends an empty resource request ([]) to the autoscaling coordinator. This should be changed to request the previous allocation instead, preserving the dataset's current resource footprint.

Related issues

Closes #60191

Additional information

  • Send current resources rather than [] when not scale up
  • Add test
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
@bveeramani bveeramani marked this pull request as ready for review January 20, 2026 23:03
@bveeramani bveeramani requested a review from a team as a code owner January 20, 2026 23:03
@bveeramani bveeramani enabled auto-merge (squash) January 20, 2026 23:03
@github-actions github-actions Bot added the go add ONLY when ready to merge, run all tests label Jan 20, 2026
)

# Initialization should send []
_send_resource_request.assert_called_with([])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general practice, I think we should avoid testing against implementation details, but I think it's okay in this case if there's not a clear alternative

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored the test to prevent mocking the _send_resource_request

bb4d311

@github-actions github-actions Bot disabled auto-merge January 21, 2026 00:30
@ray-gardener ray-gardener Bot added data Ray Data-related issues community-contribution Contributed by the community labels Jan 21, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Comment thread python/ray/data/tests/test_default_cluster_autoscaler_v2.py
bveeramani and others added 5 commits January 20, 2026 23:48
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
@bveeramani bveeramani enabled auto-merge (squash) January 23, 2026 21:26
@bveeramani bveeramani merged commit 8e120db into ray-project:master Jan 23, 2026
7 checks passed
zzchun pushed a commit to antgroup/ant-ray that referenced this pull request Jan 29, 2026
… resources when not scaling up (ray-project#60321)

## Description

When the DefaultAutoscalerV2 decides not to scale up (e.g., utilization
is low), it currently sends an empty resource request ([]) to the
autoscaling coordinator. This should be changed to request the previous
allocation instead, preserving the dataset's current resource footprint.

## Related issues
Closes ray-project#60191

## Additional information
- Send current resources rather than [] when not scale up
- Add test

---------

Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Co-authored-by: Balaji Veeramani <balaji@anyscale.com>
Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu>
zzchun pushed a commit to antgroup/ant-ray that referenced this pull request Feb 5, 2026
… resources when not scaling up (ray-project#60321)

## Description

When the DefaultAutoscalerV2 decides not to scale up (e.g., utilization
is low), it currently sends an empty resource request ([]) to the
autoscaling coordinator. This should be changed to request the previous
allocation instead, preserving the dataset's current resource footprint.

## Related issues
Closes ray-project#60191

## Additional information
- Send current resources rather than [] when not scale up
- Add test

---------

Signed-off-by: machichima <nary12321@gmail.com>
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Co-authored-by: Balaji Veeramani <balaji@anyscale.com>
Co-authored-by: Balaji Veeramani <bveeramani@berkeley.edu>
zzchun added a commit to antgroup/ant-ray that referenced this pull request Apr 15, 2026
aa640334e16 # Add util package
c535e50e365 # [Data] - Add support for callable classes for UDFExpr (ray-project#56725)
0266765ec39 # [Data] Remove constructor from ClusterAutoscaler base class (ray-project#59733)
03416b2e69f # [Data] DefaultAutoscalerV2 doesn't scale nodes from zero (ray-project#59896)
207295e23db # [Data] Introduce seams to DefaultAutoscaler2 to make it more testable (ray-project#59933)
c5eceb4ce7a # [Data] DefaultClusterAutoscalerV2 raises KeyError: 'CPU' Fix (ray-project#60208)
0bb15c93e5c # [data] Allow configuring DefaultClusterAutoscalerV2 thresholds via en… (ray-project#60133)
e52695c23df # [Data] Fix autoscaler to respect user-configured resource limits (ray-project#60283)
15c929bbc21 # [Data] Reorganize sections of `DefaultAutoscalerV2` constructor (ray-project#60347)
b5ffc068613 # [Data] Fix autoscaler to request previous allocation instead of empty resources when not scaling up (ray-project#60321)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Contributed by the community data Ray Data-related issues go add ONLY when ready to merge, run all tests

2 participants