pyspark.resource.TaskResourceRequests

class pyspark.resource.TaskResourceRequests(_jvm: Optional[py4j.java_gateway.JVMView] = None, _requests: Optional[Dict[str, pyspark.resource.requests.TaskResourceRequest]] = None)

A set of task resource requests. This is used in conjunction with the pyspark.resource.ResourceProfileBuilder to programmatically specify the resources needed for an RDD that will be applied at the stage level.

Notes

This API is evolving.

Methods

cpus(amount)
resource(resourceName, amount)

Attributes

requests

previous

pyspark.resource.TaskResourceRequest