The Anyscale scheduler acquires and distributes compute across your many Ray clusters, using your compute estate. It decides which workloads get compute, in what order, and on which resources, so a fixed pool of expensive accelerators can be shared across many teams with predictable priority and fairness.

An admin writes one declarative scheduler config. Developers keep submitting jobs, services, and workspaces the same way. Jobs and workspaces can set a priority to influence ordering.

The scheduler shares compute across Anyscale jobs, services, and workspaces. It supports a mix of compute types that you model as resource flavors, including capacity reservations, spot instances, and on-demand instances. It works across:

• VM-based clouds on AWS or Google Cloud
• Kubernetes-based clouds on Amazon EKS, Google GKE, Azure AKS, or neoclouds

A single resource queue can pool capacity across these clouds and clusters at once, so workloads fall back across reservations, spot, on-demand, and providers according to the quotas you set.

The scheduler configuration is modeled after Kueue, Kubernetes' job-queueing system, so resource flavors and queues look familiar if you've used it. See How this maps to Kueue.

Who it's for​

Core concepts​

A scheduler config has three main sections, all optional:

resource_flavors:   # the kinds of compute you have
resource_queues:    # pools with quotas that hold workloads
scheduling_rules:   # how workloads are routed to queues and prioritized

Resource flavor​

A named category of compute, defined by a label selector over the workload's labels, for example spot versus on-demand, or a specific GPU reservation. A workload's usage counts against the quota of the flavor it's assigned.

resource_flavors:
  - name: spot
    selector:
      - { key: market-type, operator: in, values: [SPOT] }

Within a queue's resource group, flavors are tried in listed order and a workload takes the first matching flavor that has quota, so list more specific flavors first. If no flavor matches, the workload is rejected.

Resource queue​

A named pool that holds workloads and caps their usage with quotas. A queue's quota is grouped by the resources it covers. For each flavor, you set a nominal_quota per resource.

resource_queues:
  - name: research
    preemption:
      within_resource_queue: lower_priority   # higher-priority work can evict lower
    resource_groups:
      - covered_resources: [gpu]
        flavors:
          - name: spot
            resources:
              - { name: gpu, nominal_quota: 64 }

Within a flavor, a covered resource with no nominal_quota is unlimited. Set a nominal_quota to cap it, or 0 to block it.

A queue with no resource_groups is a catch-all queue: it admits everything routed to it with no quota. With no config at all, every workload is admitted immediately.

Scheduling rule​

Routes an incoming workload to a queue and sets its priority. Each rule has a selector over the workload's labels, a target resource_queue, and an optional priority_policy. Rules are evaluated top to bottom and the first match wins. A rule with no selector is a catch-all.

scheduling_rules:
  - selector:
      - { key: team, operator: in, values: [research] }
    resource_queue: research
    priority_policy: { default: 50, min: 0, max: 100, on_violation: reject }
  - resource_queue: research        # catch-all for everything else
    priority_policy: { default: 0 }

priority_policy supplies a default priority and bounds requests to [min, max]. When a requested priority is out of range, on_violation: reject skips the rule and falls through to the next one, while on_violation: force_update clamps the priority into range. Priority is an integer from 0 to 1000. Higher is more important.

Preemption​

A queue with preemption.within_resource_queue: lower_priority lets a higher-priority workload evict lower-priority ones in the same queue to free quota.

For every field, see Scheduler config reference.

How scheduling works​

When a user submits a workload, Anyscale attaches labels to it. The scheduler evaluates scheduling rules top to bottom and routes the workload to a queue with an assigned priority. The first match wins, and a workload that matches no rule fails. The workload then waits in the queue, ordered first by priority, highest first, then by submission time in FIFO order, oldest first within the same priority. Once a matching flavor has quota, the scheduler admits the workload and runs it, preempting lower-priority work first if the queue allows it.

How this maps to Kueue​

The config mirrors Kueue, so admins who are familiar with Kueue may recognize it:

What's available​

The scheduler runs one Kueue-compatible config across both VM and Kubernetes clouds. You author, validate, and version that config through the CLI and SDK, which cover the config only. Rules route workloads across multiple queues and flavors, matching on custom tags and reserved labels. It applies integer priorities and within-queue preemption, and enforces fixed quotas per queue and flavor. Flavors carry declarative compute, including capacity reservations through advanced instance config, and pass through to Kueue or KAI on Kubernetes. A single queue pools capacity across clouds and clusters. A workload falls back across reservations, spot, and on-demand. You inspect queue state, stats, and events in the console.

Next steps​

• Get started: write and apply your first config with the CLI or SDK.
• Scheduler config reference: every field, with an annotated example.
• Migrate from machine pools to the Anyscale scheduler: move an existing machine pools setup to the scheduler.
• Troubleshoot the Anyscale scheduler: diagnose why a workload isn't admitted.