pyspark.resource.ResourceProfile¶
-
class
pyspark.resource.ResourceProfile(_java_resource_profile: Optional[py4j.java_gateway.JavaObject] = None, _exec_req: Optional[Dict[str, pyspark.resource.requests.ExecutorResourceRequest]] = None, _task_req: Optional[Dict[str, pyspark.resource.requests.TaskResourceRequest]] = None)¶ Resource profile to associate with an RDD. A
pyspark.resource.ResourceProfileallows the user to specify executor and task requirements for an RDD that will get applied during a stage. This allows the user to change the resource requirements between stages. This is meant to be immutable so user cannot change it after building.Notes
This API is evolving.
Attributes
executorResourcesidtaskResources