serverless_gpu.launcher module

Main launcher module for distributed serverless GPU compute.

This module provides the core functionality for launching and managing distributed functions on serverless GPU infrastructure. It includes:

  • The distributed decorator for executing functions on GPU resources

  • Workload submission and monitoring capabilities

  • Integration with Databricks jobs API and MLflow for tracking

  • Environment synchronization and dependency management

  • Support for multi-GPU and multi-node distributed workloads

The main entry point is the distributed decorator. Apply it to a function with @distributed(gpus=..., gpu_type=...), then deploy it on GPU by calling my_func.distributed(fn_args).

serverless_gpu.launcher.distributed(gpus, gpu_type=None, timeout=10800)

Decorator to launch a function on GPUs.

Parameters:

  • gpus (int) – Number of GPUs to use.

  • gpu_type (Optional[Union[GPUType, str]], optional) – The GPU type to use. Defaults to None (auto-detected).

  • timeout (Optional[float], optional) – Wall-clock seconds to allow local execution to run before subprocesses are terminated and LocalExecutionTimeoutError is raised. Defaults to 3 hours. Pass None to disable.