Lattice Logo
Lattice
Buy $99

Memory Calculator

Estimate. Plan. Optimize.

Accurate GPU memory calculations using formulas from the HuggingFace Ultrascaling Playbook. Avoid over-provisioning or OOM errors with instant, accurate memory estimates.

View docs Get Lattice
lattice.app/tools/memory-calculator
Memory Calculator showing GPU memory breakdown with color-coded components
Memory Calculator showing GPU memory breakdown with color-coded components

Key Capabilities

What Memory Calculator helps you accomplish.

  • Presets for Llama, Mistral, Mixtral, and Qwen architectures
  • ZeRO Stage 1/2/3 optimization calculations
  • Tensor and Pipeline parallelism memory splits
Memory Calculator advanced features

Advanced Features

Go deeper with advanced capabilities.

  • Activation checkpointing memory savings
  • FP32, BF16, FP16, and FP8 precision support
  • GPU fit status with specific recommendations

Technical Details

Everything you need to know about Memory Calculator.

Key Features

  • Presets for Llama, Mistral, Mixtral, and Qwen architectures
  • ZeRO Stage 1/2/3 optimization calculations
  • Tensor and Pipeline parallelism memory splits
  • Activation checkpointing memory savings

Capabilities

  • FP32, BF16, FP16, and FP8 precision support
  • GPU fit status with specific recommendations
View full documentation

Learn More About Memory Calculator

Explore related tools and documentation.

Browse all tools

Memory Calculator Guide

Complete documentation for using Memory Calculator effectively.

Get Full Access to All Tools

Access Memory Calculator plus 7 other tools, Sources, Lab, Studio, and more with a one-time purchase.

Get Lattice for $99 Browse All Tools