MCP App StoreThe latest from the AI and MCP ecosystem, curated daily.
June 4 saw a strong push toward the professionalization of AI evaluation and safety, with a particular focus on agentic tool-use and enterprise-grade guardrails. The most significant move comes from ServiceNow AI with EVA-Bench 2.0, which provides a massive, structured framework for measuring how agents actually perform across hundreds of real-world tool scenarios.
Alongside evaluation, NVIDIA continued to refine the Nemotron ecosystem, releasing both a multimodal content safety layer and a new synthetic data generation method to improve pretraining quality. Meanwhile, OpenAI is doubling down on long-term personalization with a new memory system for ChatGPT, aiming to reduce context loss across conversations.
Today's stories:
Overall, the theme of the day is the transition from "chatbots" to "reliable agents" through better measurement and safety.
NVIDIA releases Nemotron 3.5 Content Safety, providing customizable multimodal safety guardrails for enterprise AI. It focuses on scaling safety checks across global deployments and various data modalities.
Google has launched local development capabilities for Kaggle Benchmarks, allowing developers to create and test AI benchmarks more efficiently. This streamlines the evaluation process for AI models on local machines.
ServiceNow AI introduces EVA-Bench Data 2.0, a comprehensive dataset for evaluating agentic AI across 121 tools and 213 scenarios. This provides a robust framework for measuring tool-use and task-completion capabilities of AI agents.
NVIDIA introduces a new method for generating high-quality synthetic Q&A pairs to improve the pretraining of Nemotron models. This approach leverages task-seeded generation to create more diverse and accurate training data.
OpenAI introduces a new memory system for ChatGPT designed to better remember user preferences and maintain context across conversations. This improvement aims to make the assistant more personalized and relevant over the long term.
Hugging Face introduces a redesigned CLI optimized for AI agents to interact with the Hub. This enables more seamless model and dataset management within agentic workflows.