NeMo Megatron Reinforces NVIDIA AI Leadership in Large Language Models Alberto Romero, LLM Analyst Karl Freund, Founder and Principal Analyst Cambrian-AI Research.
updates to the NeMo Megatron framework that provides training speedups of up to 30%. This translates into a 175B-parameter model being trained in 24 days instead of 34 using 1024 A-100 GPUs.... [+]One of the updates improves training efficiency. It comprises a couple of techniques to reduce memory requirements during training that reduces computational time. Sequence parallelism reduces activation memory requirements beyond the usual tensor and pipeline parallelism methods.
NVIDIA says the combination of both techniques provides a 5x reduction in memory requirements with respect to doing only tensor parallelism. The overhead is just 2–4% in comparison to +36% in the case of applying full activation recomputation. In total, this translates into , and thus a significant reduction in training times across model sizes, from 22 billion to 1 trillion parameters.... [+]
t per-layer breakdown of forward, backward, and recompute times. Baseline is the case with no recomputation and no sequence parallelism. These techniques are effective at reducing the overhead incurred when all activations are recomputed instead of saved. For the largest models, overhead drops from 36% to just 2%.” -NVIDIAThe other update improves deployment times. It’s a hyperparameter tool to “automatically find optimal training and inference configurations.
NVIDIA is willing to work to keep its dominance regardless of how many changes occur in the AI field or how fast they happen. New companies emerge with singular ideas that may better fit current trends, but they still need to build the full stack that NVIDIA has been maturing for many years. That advantage is difficult to overcome; most startups dedicate at least 50% of their engineering staff to software.
Brasil Últimas Notícias, Brasil Manchetes
Similar News:Você também pode ler notícias semelhantes a esta que coletamos de outras fontes de notícias.
Google fires employee who claimed AI system had become sentientThe software engineer claimed the conversation technology had reached a level of consciousness after he exchanged thousands of messages with it.
Consulte Mais informação »
‘The entire protein universe’: AI predicts shape of nearly every known proteinDeepMind’s AlphaFold tool has determined the structures of around 200 million proteins, from almost every known organism on Earth.
Consulte Mais informação »
‘The entire protein universe’: AI predicts shape of nearly every known proteinDeepMind’s AlphaFold tool has determined the structures of around 200 million proteins, from almost every known organism on Earth.
Consulte Mais informação »
GFP-GAN is a New Free AI Tool That Can Fix Most Old Photos InstantlyA new AI tool called GFPGAN can instantly restore old damaged and faded photos in a surprisingly realistic way.
Consulte Mais informação »
AI Ethics And That Viral Story Of The Chess Playing Robot That Broke The Finger Of A Seven-Year-Old During A Heated Chess Match Proffers Spellbinding Autonomous Systems LessonsAI Ethics issues arise in a recent viral news story about a young boy playing tournament chess that got his finger broken by a chess playing robotic arm.
Consulte Mais informação »