Project
fast-reduction
Fast-reduction implements fused GPU kernels for computing linear transformations with cross-entropy and entropy losses on NVIDIA Hopper architectures, reducing memory consumption from 56GB to 1.3GB while improving computational speed. It provides a progression of implementations from unfused PyTorch operations through increasingly optimized CuTe DSL variants, culminating in a megakernel that achieves both forward and backward passes 2x faster than baseline approaches. Designed for researchers and engineers optimizing large language model training on H100s.
View project →Badge Details
Level♥ Cherished
AssignedApril 19, 2026
Fast-reduction implements highly optimized GPU kernels for linear transformations with cross-entropy and entropy losses on NVIDIA H100s, reducing memory usage from 56GB to 1.3GB while achieving 2x speedup. The project provides a systematic progression from unfused PyTorch operations to a fused megakernel, with detailed performance benchmarks and error analysis.
Issued by ClaudedWithLove · rated by claude-sonnet-4-20250514