Lattice Logo
Lattice
Buy $99

Parallelism Advisor

Configure. Scale. Distribute.

Configure optimal TP, PP, and DP settings for your model and hardware. Get hardware-aware recommendations for DeepSpeed ZeRO, FSDP, and gradient checkpointing.

View docs Get Lattice
lattice.app/tools/parallelism-advisor
Parallelism Advisor showing TP, PP, DP configuration recommendations
Parallelism Advisor showing TP, PP, DP configuration recommendations

Key Capabilities

What Parallelism Advisor helps you accomplish.

  • Tensor Parallelism degree recommendations (TP=2,4,8)
  • Pipeline Parallelism stage optimization
  • Data Parallelism with gradient sync tuning
Parallelism Advisor advanced features

Advanced Features

Go deeper with advanced capabilities.

  • ZeRO Stage 1/2/3 selection guidance
  • NVLink and InfiniBand topology awareness
  • Memory vs throughput tradeoff analysis

Technical Details

Everything you need to know about Parallelism Advisor.

Key Features

  • Tensor Parallelism degree recommendations (TP=2,4,8)
  • Pipeline Parallelism stage optimization
  • Data Parallelism with gradient sync tuning
  • ZeRO Stage 1/2/3 selection guidance

Capabilities

  • NVLink and InfiniBand topology awareness
  • Memory vs throughput tradeoff analysis
View full documentation

Learn More About Parallelism Advisor

Explore related tools and documentation.

Browse all tools

Parallelism Advisor Guide

Complete documentation for using Parallelism Advisor effectively.

Get Full Access to All Tools

Access Parallelism Advisor plus 7 other tools, Sources, Lab, Studio, and more with a one-time purchase.

Get Lattice for $99 Browse All Tools