ray-train
Category: ml-training · License: MIT · Version: 1.0.0
Tags: Ray Train, Distributed Training, Orchestration, Ray, Hyperparameter Tuning, Fault Tolerance, Elastic Scaling, Multi-Node, PyTorch, TensorFlow
When to use it
Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.Example requests
Ask your agent in plain language, for example:- “How do I run my PyTorch training across 4 machines with 8 GPUs each?”
- “My train.py fine-tunes a Llama-style model with HF Trainer on a single A100 and I now have a 6-node Ray cluster. Running ray status shows only the head node, and workers fail with ‘Unable to connect to GCS at 10.0.0.5:6379’. How do I get this training across the whole cluster with checkpointing so it survives a node dying?”
- “I need to find the best learning rate, weight decay and batch size for my ResNet on our shared cluster. I’d like to try a couple hundred configs in parallel and stop the bad ones early. What’s the approach?”
Install
/plugin marketplace add KalarisLabs/research-agent-skills and install the whole ml-training category with /plugin install ml-training@research-agent-skills.
Related
daskpytorch-lightning-distributedray-dataweights-and-biasesskypilot-multi-cloud-orchestration- Skills for machine learning research