[rllib] Parallel-data loading and multi-gpu support for IMPALA by ericl · Pull Request #2766 · ray-project/ray

ericl · 2018-08-29T19:21:56Z

What do these changes do?

Add support for multi-gpu optimizer to IMPALA.
Add option for parallel data loading and replay.
These are all disabled by default.

Related issue number

AmplabJenkins · 2018-08-29T19:45:53Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/7862/
Test FAILed.

AmplabJenkins · 2018-08-29T20:04:06Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/7863/
Test FAILed.

AmplabJenkins · 2018-09-03T04:36:49Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/7951/
Test FAILed.

AmplabJenkins · 2018-09-03T04:52:58Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/7952/
Test FAILed.

ericl

Personally, I think the multi-GPU stuff is too messy and we should move away from it. However it's not clear what the best way to do that is, perhaps with TF eager, or moving a lot of the logic into numpy, it would be simpler.

ericl · 2018-10-14T00:15:11Z

        return Resources(
            cpu=1,
-            gpu=cf["gpu"] and cf["gpu_fraction"] or 0,
+            gpu=cf["num_gpus"] and cf["num_gpus"] * cf["gpu_fraction"] or 0,


I don't know what will happen if gpu_fraction > 1, but for < 1 I believe it should work.

ericl · 2018-10-14T00:20:50Z

Also, having public documentation for the shared configs and algorithm specific things would be really good.

It would be good to figure this out. Sphinx doesn't seem to have a good way to do this, unless you duplicate the comments in the dict, or change it to a class or something. For now I think it's reasonable to expect users to read the code since it's at the top of the file.

AmplabJenkins · 2018-10-14T00:38:49Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8626/
Test FAILed.

AmplabJenkins · 2018-10-14T00:44:03Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8625/
Test FAILed.

AmplabJenkins · 2018-10-14T01:38:48Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8629/
Test FAILed.

ericl · 2018-10-14T02:49:44Z

I just did a quick multi-GPU run on PPO / IMPALA atari and perf looks good still.

AmplabJenkins · 2018-10-14T03:33:49Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8631/
Test FAILed.

AmplabJenkins · 2018-10-14T05:07:55Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8637/
Test FAILed.

AmplabJenkins · 2018-10-14T05:34:58Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8638/
Test FAILed.

AmplabJenkins · 2018-10-14T07:25:36Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8642/
Test FAILed.

AmplabJenkins · 2018-10-14T08:07:39Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8644/
Test FAILed.

AmplabJenkins · 2018-10-14T08:16:44Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8643/
Test FAILed.

AmplabJenkins · 2018-10-15T06:22:00Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8653/
Test FAILed.

AmplabJenkins · 2018-10-15T08:51:05Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8661/
Test PASSed.

ericl added 15 commits August 18, 2018 14:56

wip

33f01bb

stats

d6b5b5d

fix assert

56081a2

fix par opt

33c4871

clip

10cb4ac

fix

517b1ab

gpu queu

c84906e

multiple loader threadsst

0a01023

thread safe

a33bff0

fix

50843da

icr threads

763f165

depth

2d4e332

Merge remote-tracking branch 'upstream/master' into impala-multigpu

e5f9095

revert local

91df50b

fix

60f2d4b

ericl added 10 commits August 29, 2018 16:48

frac gpu

9676459

doc

84cc7db

Update rllib-training.rst

2295a15

yapf

10ad1f7

Merge branch 'test-frac-gpu' of github.com:ericl/ray into test-frac-gpu

79d9677

remove xray

ee3b599

replay params

6cc13b1

fix assert

6070872

Merge remote-tracking branch 'upstream/master' into impala-multigpu

e6fe182

rep

0c4db6d

note

54acce9

ericl commented Oct 14, 2018

View reviewed changes

comments 0

0dd2bae

fix lstm case

9d2b535

ericl added 2 commits October 13, 2018 20:55

set to 99999

daebf65

fix assert

8d66ded

doc

8979a54

ericl added 4 commits October 14, 2018 00:40

impala lstm test

7c81fe9

remove unneeded assert

d94fe31

ppo mask

bae28f6

doc

76640e3

richardliaw approved these changes Oct 14, 2018

View reviewed changes

ericl added 3 commits October 14, 2018 22:08

lint

507ae93

fix mask

73005ca

fix non lstm case

6b3f6f3

fix impala

c7d893d

damnit flake8

b76c0c4

ericl merged commit 3c891c6 into ray-project:master Oct 15, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[rllib] Parallel-data loading and multi-gpu support for IMPALA#2766

[rllib] Parallel-data loading and multi-gpu support for IMPALA#2766
ericl merged 51 commits into
ray-project:masterfrom
ericl:impala-multigpu

ericl commented Aug 29, 2018 •

edited

Loading

AmplabJenkins commented Aug 29, 2018

AmplabJenkins commented Aug 29, 2018

AmplabJenkins commented Sep 3, 2018

AmplabJenkins commented Sep 3, 2018

ericl left a comment

ericl Oct 14, 2018

ericl commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

ericl commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 15, 2018

AmplabJenkins commented Oct 15, 2018

Labels

3 participants

Uh oh!

Conversation

ericl commented Aug 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What do these changes do?

Related issue number

AmplabJenkins commented Aug 29, 2018

AmplabJenkins commented Aug 29, 2018

AmplabJenkins commented Sep 3, 2018

AmplabJenkins commented Sep 3, 2018

ericl left a comment

Choose a reason for hiding this comment

ericl Oct 14, 2018

Choose a reason for hiding this comment

ericl commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

ericl commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 14, 2018

AmplabJenkins commented Oct 15, 2018

AmplabJenkins commented Oct 15, 2018

Labels

3 participants

ericl commented Aug 29, 2018 •

edited

Loading