A team of researchers from the Allen Institute for AI, Contextual AI, and the University of Washington have released OLMoE (Open Mixture-of-Experts Language Models), a new open-source LLM that achieves state-of-the-art performance while using significantly fewer computational resources than comparable models. OLMoE utilizes a Mixture-of-Experts (MoE) architecture, allowing it to have 7 billion total parameters […]

The post OLMoE Achieves State-Of-The-Art Performance using Fewer Resources and MoE appeared first on AIM.