Hi!
Thank you for creating TPOT!
I'm using TPOT==0.12.2.
My server has 96 cores and I always notice that TPOT uses all cores only for first 5-7 minutes of each "generation" (right?). And after that TPOT uses only 1 core for next few hours.
I tried to limit that process life time using different combinations of arguments:
- generations = None
- max_time_mins = 30
- max_eval_time_mins = 30
For the first time I use to think that "max_time_mins" works like "signal" in python and rises an exception to stop a bench of processes that you call "generation". But it doesn't. And I don't understand why.
Could you tell me what combination of arguments should I set to stop one "generation" in 30 minutes?
P.S.
I've created a custom metric to explore the problem and found strange situations. For example. TPOT has found a good pipeline at the beginning of process (within first 5 minutes after start when 96 cores worked hard). Metric of that pipeline was printed and it was "perfect" for me. But after that all 95 process was finished and 1 process worked for next hour. After an hour TPOT finished all jobs and returned to fitted_pipeline_ NOT a "perfect" pipeline. But a "random" (?) or last(?) pipeline with very bad score. I wanted to get "perfect" pipeline (with printed score). But I received not a good pipeline. Why? If it is an error it is probably connected with that 1 super long process.
P.P.S.
It's very painful to look at one working core for few hours:
Hi!
Thank you for creating TPOT!
I'm using TPOT==0.12.2.
My server has 96 cores and I always notice that TPOT uses all cores only for first 5-7 minutes of each "generation" (right?). And after that TPOT uses only 1 core for next few hours.
I tried to limit that process life time using different combinations of arguments:
For the first time I use to think that "max_time_mins" works like "signal" in python and rises an exception to stop a bench of processes that you call "generation". But it doesn't. And I don't understand why.
Could you tell me what combination of arguments should I set to stop one "generation" in 30 minutes?
P.S.
I've created a custom metric to explore the problem and found strange situations. For example. TPOT has found a good pipeline at the beginning of process (within first 5 minutes after start when 96 cores worked hard). Metric of that pipeline was printed and it was "perfect" for me. But after that all 95 process was finished and 1 process worked for next hour. After an hour TPOT finished all jobs and returned to fitted_pipeline_ NOT a "perfect" pipeline. But a "random" (?) or last(?) pipeline with very bad score. I wanted to get "perfect" pipeline (with printed score). But I received not a good pipeline. Why? If it is an error it is probably connected with that 1 super long process.
P.P.S.
It's very painful to look at one working core for few hours: