Add pipe receiver memory protection#18090
Conversation
Caideyipi
force-pushed
the
codex/pipe-receiver-memory-protection
branch
from
e8a0833 to
78ced86
Compare
jt2594838
left a comment
jt2594838
left a comment
There was a problem hiding this comment.
It is worth noticing that for come-and-go requests, it already holds the Thrift FrameBuffer.
Therefore, if it is not cached by Pipe or uncompressed, there is no necessity to allocate Pipe memory for it.
Even if it is uncompressed, you may only need to allocate the part that exceeds the FrameBuffer.
| return new TPipeTransferResp( | ||
| getReceiverTemporaryUnavailableStatus( | ||
| "decompressing pipe transfer request", requestedMemorySizeInBytes, e)); |
There was a problem hiding this comment.
Applied in 9213e48: the receiver OOM status text now uses the DataNode pipe i18n messages with English and Chinese entries.
| } catch (Exception ex) { | ||
| return PipeTransferFilePieceResp.toTPipeTransferResp(status); | ||
| } |
There was a problem hiding this comment.
The exception is not reflected in the status?
There was a problem hiding this comment.
Applied in 9213e48: the returned status now keeps the receiver OOM context and includes the root-cause exception message instead of dropping it.
2 participants
Description
Tests