Archives for Microsoft DeepSpeed
Over the years, many open-source deep learning optimisation libraries have been announced by tech giants but DeepSpeed remains one of the most popular.
10
Feb
Microsoft, NVIDIA test waters for a large-scale generative language model with promising results
We believe that our results and findings can help, shape, and facilitate future research in foundational, large-scale pretraining.
The key is that 1T was never ‘trained to convergence.’
MT-NLG has 3x the number of parameters compared to the existing largest models – GPT-3, Turing NLG, Megatron-LM and others.
15
Sep
Microsoft Releases Latest Version Of DeepSpeed, Its Python Library For Deep Learning Optimisation
Recently, Microsoft announced the new advancements in the popular deep learning optimisation library known as DeepSpeed. This library is an important part of Microsoft’s new AI at Scale initiative to enable next-generation AI capabilities at scale. DeepSpeed, the open-source deep learning training optimisation library was unveiled in February this year along with ZeRO (Zero Redundancy…
The post Microsoft Releases Latest Version Of DeepSpeed, Its Python Library For Deep Learning Optimisation appeared first on Analytics India Magazine.