PyTorch/Hugging Face batching utility that sorts variable-length text by difficulty, then dynamically increases batch size on easier samples using a pre-trained VRAM predictor to improve GPU utilization and throughput while reducing OOM risk with fallback handling.
machine-translation transformers pytorch dataloader sampler huggingface dynamic-batching vram-optimization huggingface-trainer
-
Updated
Apr 28, 2026 - Python