Microsoft AI & Research has created the largest Transformer-based language generation model named Turing NLG, with 17 billion parameters, surpassing Nvidia Megatron and OpenAI’s GPT-2 in size and achieving state-of-the-art results in NLP tasks. Initially available in private demos, Turing NLG is set to revolutionize language generation models. Additionally, Microsoft has open-sourced DeepSpeed, a deep learning library designed to optimize distributed training of large models, such as Turing NLG, by providing high-performance and low-latency inference capabilities.
The field of NLP models saw significant advancements in 2019 with the Transformer architecture, leading to models like Roberta, Google XLNet, and Microsoft MT-DNN. Language generation models using Transformers have demonstrated the ability to predict next words, generate responses, and summarize text effectively. With the release of DeepSpeed and ZeRO Optimizer, Microsoft aims to provide developers and machine learning professionals with tools to train large networks efficiently and at scale, addressing the cost and scalability challenges associated with training models using the Transformer architecture.
In other news, Google DeepMind introduced the Compressive Transformer and PG19 benchmark for long-range memory modeling and language generation analysis, showcasing the ongoing advancements in the field of natural language artificial intelligence. The landscape of AI continues to evolve rapidly, with new breakthroughs and innovations shaping the future of language generation and NLP tasks.
As the industry gathers for VB Transform 2024 in San Francisco, business leaders have the opportunity to connect, explore generative AI applications, and learn how to integrate AI technologies into their respective industries. The event promises to be a platform for sharing insights, networking with peers, and embracing the challenges and opportunities presented by AI advancements. Don’t miss the chance to be part of this transformative AI event and stay informed with the latest developments in the AI industry.
Article Source
https://venturebeat.com/ai/microsoft-trains-worlds-largest-transformer-language-model/