[Ray Data] Improve the design and documentation of _MapWorker
#46323
Labels
data
Ray Data-related issues
enhancement
Request for new feature and/or capability
P1
Issue that should be fixed within a few weeks
_MapWorker
#46323
Description
Background:
_MapWorker
are actors created byActorPoolMapOperator
, which is used in the case where users call map-like APIs with a callable classUDF
.Use case
I have a ray cluster that is stuck attempting to schedule a
_MapWorker
. This is executing a complex ray data pipeline which contains more than one UDF, so a few problems arise:_MapWorker
and the user has to resolve the "optimal ray data DAG execution" in their head to guess what might be the issue._MapWorker
does given the name on its own is not self-explicableSee the attached screenshot for more details
Improvement suggestions:
_MapWorker
construct or whatever we end up naming it does, and what does its methods like__init__
andget_location
do..._MapWorker
to something more public facing likeLaunchWorker
LaunchWorker
under a subgroup of theMapWorker
it is trying to createThe text was updated successfully, but these errors were encountered: