Move /rllib/ into /python/ray/ - #66096
Open
pseudo-rnd-thoughts wants to merge 6 commits into
Open
pseudo-rnd-thoughts wants to merge 6 commits into
pseudo-rnd-thoughts wants to merge 6 commits into
Conversation
Signed-off-by: Mark Towers <mark@anyscale.com>
pseudo-rnd-thoughts
added
rllib
pseudo-rnd-thoughts
marked this pull request as ready for review
September 11, 2026 15:24
pseudo-rnd-thoughts
requested review from
a team,
edoakes and
richardliaw
as code owners
September 11, 2026 15:24
![gemini-code-assist[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request relocates the rllib directory from the repository root to python/ray/rllib/ and updates all corresponding references across build configurations, CI scripts, and documentation. The review comments correctly identify a few minor issues in the updated files, including an incorrect implementation path for the APPO algorithm and the use of /tree/master/ instead of /blob/master/ for file links in the RLlib examples documentation.
added 2 commits
September 14, 2026 10:17
Signed-off-by: Mark Towers <mark@anyscale.com>
Contributor
|
Thanks for doing this! |
Contributor
|
@pseudo-rnd-thoughts — I went through this from the docs side and it looks good. The doc changes are mechanical path plumbing and I couldn't find anything that breaks:
This is static verification rather than a full docs build, but it covers the breakage a directory move introduces. One heads-up: I have a stack of RLlib changes in flight. Once those merge down, these doc path references will likely need a refresh, since my work will almost certainly introduce merge conflicts with this PR. How would you like to handle that? Two options:
Happy either way — let me know which you'd prefer. Docs-side verification drafted with Claude Code assistance; I've reviewed it and stand behind it. |
Signed-off-by: Mark Towers <mark@anyscale.com>
Member
Author
If you've got the PRs ready to go then lets do 1 |
Contributor
|
Hi @pseudo-rnd-thoughts, heads-up: master converted doc pages this PR edits from reStructuredText to MyST Markdown, so those edits need to move to the
If you'd like help translating your edits to Markdown, DM me (Douglas Strodtman) on the Ray Slack and I'll help. If you'd rather handle it yourself, the repo has a Claude Code skill for this conversion at |
dstrodtman
added a commit
that referenced
this pull request
Sep 28, 2026
…ervability pages to MyST (#66461) ## Why are these changes needed? Converts the last non-API-reference RST pages under `ray-core/`, `data/`, and `rllib/` to MyST Markdown, plus the `train/` and `ray-observability/` pages that `doc/BUILD.bazel` names by path. The API reference trees (`*/api/`, `rllib/package_ref/`, and `ray-core/compiled-graph/compiled-graph-api`) stay RST for now. Each page gets its own commit, so you can review one file at a time. Every commit renames the page's literal `.rst` paths in `doc/BUILD.bazel` to `.md`. These paths are in `exclude` lists and in the explicit `files` lists of `doctest[data-gpu]` and `doctest[train-gpu]`, and they don't glob. Without the edit, an excluded page would silently join its target once it's `.md`, and an included page would silently drop out of it. Conversions are format-only: labels, headings, and prose are unchanged. A final whitespace-only commit soft-wraps the prose. `ray-soft-wrap`'s `verify.py` confirms non-whitespace bytes, rendered HTML, and idempotence for all 11 files. ### `rllib/getting-started`: doctest target removed The first commit deletes `doctest[rllib2]`, the only target over `rllib/getting-started.rst`, before the page converts. The target is tagged `manual` with a hang TODO, so no premerge or postmerge step has run it, and nothing else references it (`git grep rllib2` is empty). Its 15 `testcode` blocks had no `testoutput`, so they asserted nothing. The 12 visible blocks become display-only `python` code blocks. The three `:hide:` blocks held `.stop()` cleanups that never rendered, so they're deleted. `doctest[rllib]` still excludes the page. If these examples should be validated later, a `doc_code/` script is the better route than reviving the hanging target. cc @elliot-barn, who owns the TODO. This commit isn't buildable on its own. It turns a `.. testcode::` that had no blank line before its body into a `.. code-block:: python`, still with no blank line, and docutils reads the body as extra arguments to the directive. The later conversion commit fixes it in the `.md`, and a squash merge never puts the intermediate commit on master. ### Doctest coverage Docs-only PRs don't run doctests premerge. This PR edits `doc/BUILD.bazel`, so every library's docs example step runs, but premerge can't show the conversion kept the same blocks. For that, I parsed each `.md` and the `.rst` it replaced with Ray's `pytest-sphinx` fork, the same parser the doctest macro runs, and asserted identical sections (directive, body, `:options:`, `:skipif:`) and examples. | Page | BUILD role | Examples before → after | | --- | --- | --- | | `ray-core/handling-dependencies` | `doctest[core]` exclude | 17 → 17 | | `ray-core/tasks/nested-tasks` | `doctest[core]` exclude | 0 → 0 (`literalinclude` of a tested `doc_code/` file) | | `data/batch-inference` | `doctest_each` exclude, `doctest[data-gpu]` files | 7 → 7 | | `data/transforming-data` | `doctest_each` exclude, `doctest[data-gpu]` files | 24 → 24 | | `data/contributing/contributing` | `doctest_each` glob | 0 → 0 (toctree only) | | `rllib/getting-started` | `doctest[rllib]` exclude, `doctest[rllib2]` removed | 15 → 0, by the removal commit | | `ray-observability/user-guides/cli-sdk` | top-level exclude | 16 → 16 | | `ray-observability/user-guides/ray-tracing` | top-level exclude | 3 → 3 | | `train/horovod` | `doctest[train]` exclude | 2 → 2 | | `train/user-guides/data-loading-preprocessing` | `doctest[train]` exclude, `doctest[train-gpu]` files | 6 → 6 | | `train/user-guides/using-accelerators` | `doctest[train]` exclude, `doctest[train-gpu]` files | 10 → 10 | To show the check would catch a broken conversion, I mutated each page with examples three ways: dropping a `testcode` block, altering a line in one, and turning one into a four-backtick fence. The last renders fine but is never collected. The check failed on all 24 mutations. ### Rendering changes I ran `render_diff.py` on all 11 pages against `/en/master`. Six render identically: `nested-tasks`, `data/contributing/contributing`, `cli-sdk`, `ray-tracing`, `horovod`, and `using-accelerators`. Every difference on the other five is listed here. - `rllib/getting-started`: on master, one `.. testcode::` line had no blank line after it. docutils read the block's first six lines, the `best_result = results.get_best_result(...)` call, as the directive's argument and dropped them. So master shows code that uses `best_result` without showing where it comes from. The converted page renders all six lines. This is the one place the conversion deliberately doesn't reproduce master. - `rllib/getting-started`: the "Farama gymnasium" link now has `https://`. On master it's a relative link to `gymnasium.farama.org` that 404s. - `ray-core/handling-dependencies`: MyST drops the apostrophe when it slugifies the "can't import the packages" heading, so the section id changes. That heading had no label, so the page adds a target carrying the old docutils slug, and existing `#...-can-t-import-...` links still resolve. An auto-numbered span id, which nothing links to, also changes from `id4` to `id2`. - `data/batch-inference` and `train/user-guides/data-loading-preprocessing`: each renders an RST comment as an HTML comment, which readers never see. - `data/transforming-data`: the "When using" definition list's `<dl>` gains the unstyled `myst` class. The first preview diff also caught three regressions that the green build didn't. MyST's `replacements` extension turned two `(c)` list markers into `©` and an `etc..` into `etc…`. The anchor above had changed too. A follow-up commit fixes all three. A sweep of the 11 pages found no other text that extension rewrites. ### Dead `doc/BUILD.bazel` excludes A final commit drops four `exclude` entries that name files that have moved: two in `doctest[serve]` and two in the serve `doc_code` test. A glob exclude naming a missing file excludes nothing, so these had silently stopped applying. This doesn't change what any target runs. #66462 carries the same commit, along with a lint check that fails on this class of stale path, so the two PRs merge cleanly in either order. ### Open PRs that edit these files Each of these open PRs edits one of the converted `.rst` files, so each will need to move its change to the `.md`: #66279, #66191, #65956, #65933, and #65830 (`handling-dependencies`), #65693 (`cli-sdk`), and #66419, #66420, #66422, and #66423 (`data/batch-inference`, `train/user-guides/data-loading-preprocessing`). #66096, #64053, and #63881 also edit `doc/BUILD.bazel`, and only #66096 edits nearby lines. ## Related issue number None. No open PR converts these pages. I checked `gh pr list` for open PRs touching each file. ## Checks - [x] I've signed off every commit (`git commit -s`). - [x] Pre-commit hooks passed on every commit. - Doctest parity: `python3 doctest_parity.py <the 11 .md files>` with `pytest==7.4.4` and `pytest-sphinx @ git+https://github.com/ray-project/pytest-sphinx`. All OK, run again after the soft-wrap commit. - Static checks on every `.md`: fences balance, no leftover RST roles or directives, every label from the `.rst` is present, and directive counts match. - Read the Docs preview builds green under `fail_on_warning`, and the render diff against `/en/master` is in Rendering changes. The first build hit the known autosummary-import segfault and passed on rebuild. - Local doctest run: not done. The GPU targets need hardware, and the CPU targets run in this PR's premerge because of the `BUILD.bazel` change. AI assistance (Claude Code) was used for the conversion and the verification scripts. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Douglas Strodtman <douglas@anyscale.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ython-ray Signed-off-by: Mark Towers <mark@anyscale.com> # Conflicts: # .github/CODEOWNERS # doc/source/rllib/algorithm-config.rst # doc/source/rllib/checkpoints.rst # doc/source/rllib/env-to-module-connector.rst # doc/source/rllib/external-envs.rst # doc/source/rllib/hierarchical-envs.rst # doc/source/rllib/index.rst # doc/source/rllib/key-concepts.rst # doc/source/rllib/learner-connector.rst # doc/source/rllib/metrics-logger.rst # doc/source/rllib/multi-agent-envs.rst # doc/source/rllib/new-api-stack-migration-guide.rst # doc/source/rllib/rl-modules.rst # doc/source/rllib/rllib-advanced-api.rst # doc/source/rllib/rllib-algorithms.rst # doc/source/rllib/rllib-callback.rst # doc/source/rllib/rllib-dev.rst # doc/source/rllib/rllib-env.rst # doc/source/rllib/rllib-examples.rst # doc/source/rllib/rllib-fault-tolerance.rst # doc/source/rllib/rllib-offline.rst # doc/source/rllib/rllib-replay-buffers.rst # doc/source/rllib/scaling-guide.rst # pyproject.toml # python/ray/rllib/README.md # python/ray/rllib/README.rst
elliot-barn
approved these changes
Sep 30, 2026
ArturNiederfahrenhorst
approved these changes
Oct 1, 2026
ArturNiederfahrenhorst
left a comment
ArturNiederfahrenhorst
left a comment
Contributor
There was a problem hiding this comment.
Awesomeeeee!
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Over 7 years ago
rllibwas moved frompython/ray/rllibto the root folder where its lived ever since (#5324).However, as Ray has grown and rllib positions in the root folder makes less sense and causes more issues for CI and docs to maintain due to its "weird" position relative to the other Ray projects.
This PR moves
rllibback topython/ray/rllibupdating all the relevant documentation and CIFor users, this should be a primarily internal change and no external users being aware of the changes (other than PRs)