pytorch-lightning-distributed
Category: ml-training · License: MIT · Version: 1.0.0
Tags: PyTorch Lightning, Training Framework, Distributed Training, DDP, FSDP, DeepSpeed, High-Level API, Callbacks, Best Practices, Scalable
When to use it
High-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.Example requests
Ask your agent in plain language, for example:- “How do I turn my plain PyTorch training loop into a Lightning Trainer setup with DDP on 4 GPUs?”
- “I’m training a ResNet-50 on my cell-imaging dataset (train.py) on a 2-node cluster, 8 A100s each. Job launched with sbatch but I get ‘RuntimeError: Timed out initializing process group in store based barrier’ and Lightning seems to spawn extra processes. How do I set up the Trainer so it works across both nodes?”
- “I want my model training to automatically save the 3 best checkpoints by validation loss, stop if it hasn’t improved in 5 epochs, and log the learning rate, without writing all that loop code myself. What’s the cleanest way to do it in PyTorch?”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole ml-training category with /plugin install ml-training@research-agent-skills.
Related
huggingface-acceleratepytorch-lightningray-traintorchforge-rl-training- Skills for machine learning research