Mixtral

Mistral AI's 141B 8x22B MoE base model — significantly stronger than 8x7B with 39B active params, matching GPT-3.5 on most benchmarks.

Mistral AI's landmark 8x7B MoE instruct model — sparse mixture-of-experts with GPT-3.5-level performance at fraction of the compute.