As AI agents connect to more services—Slack, GitHub, Jira, MCP servers—tool libraries grow rapidly. A typical multi-server setup can easily have 50+ tools consuming 55,000+ tokens before any conversation starts. Worse, tool selection accuracy degrades when models face 30+ similarly-named tools. The Tool Search Tool pattern, pioneered by Anthropic, addresses this: instead of loading all tool definitions upfront, the model discovers tools on-demand. It receives only a search tool initially, queries for capabilities when needed, and gets relevant tool definitions expanded into context. This…
JSON is the go-to format for LLM tool responses, but recent discussions around alternative formats like TOON (Token-Oriented Object Notation) claim potential benefits in token efficiency and performance.
While the debate continues—with critical analyses pointing to context-dependent results—the question is: how to experiment with these formats in your own Spring AI applications? This article demonstrates how to configure Spring AI to convert tool responses between JSON, TOON, XML, CSV, and YAML, enabling you to decide what works best for your specific use case. Spring AI Tool Calling: A Quick…
The challenge of evaluating Large Language Model (LLM) outputs is critical for notoriously non-deterministic AI applications, especially as they move into production. Traditional metrics like ROUGE and BLEU fall short when assessing the nuanced, contextual responses that modern LLMs produce.
Human evaluation, while accurate, is expensive, slow, and doesn't scale. Enter LLM-as-a-Judge - a powerful technique that uses LLMs themselves to evaluate the quality of AI-generated content. Research shows that sophisticated judge models can align with human judgment up to 85%, which is actually higher…
The Spring AI ChatClient offers a fluent API for communicating with an AI model.
The fluent API provides methods for building the constituent parts of a prompt that gets passed to the AI model as input. Advisors are a key part of the fluent API that intercept, modify, and enhance AI-driven interactions.
The key benefits include encapsulating common Generative AI patterns, transforming data sent to and from Large Language Models (LLMs), and providing portability across various models and use cases. Advisors process ChatClientRequest and ChatClientResponse objects.
The framework chains advisors…
The Model Context Protocol (MCP) standardizes how AI applications interact with external tools and resources.
Spring joined the MCP ecosystem early as a key contributor, helping to develop and maintain the official MCP Java SDK that serves as the foundation for Java-based MCP implementations.
Building on this contribution, Spring AI has embraced MCP with comprehensive support through dedicated Boot Starters and MCP Java Annotations, making it easier than ever to build sophisticated AI-powered applications that can seamlessly connect to external systems. This blog introduces core MCP components…
The Model Context Protocol (MCP) is a powerful feature in Spring AI that enables AI models to access external tools and resources through a standardized interface. One interesting capabilities of MCP is its ability to dynamically update available tools at runtime. This blog post explores how Spring AI implements dynamic tool updates in MCP, providing flexibility and extensibility to AI-powered applications. The related example code is available here: Dynamic Tool Update Example Understanding the Model Context Protocol Before diving into dynamic tool updates, let's understand what MCP is and…
This blog post demonstrates practical implementations of Prompt Engineering techniques using Spring AI. The examples and patterns in this article are based on the comprehensive Prompt Engineering Guide that covers the theory, principles, and patterns of effective prompt engineering. The blog shows how to translate those concepts into working Java code using Spring AI's fluent ChatClient API. For convenience, the examples are structured to follow the same patterns and techniques outlined in the original guide. The demo source code used in this article is available at: https://github.com/spring…
In a recent research publication: Building effective agents, Anthropic shared valuable insights about building effective Large Language Model (LLM) agents. What makes this research particularly interesting is its emphasis on simplicity and composability over complex frameworks. Let's explore how these principles translate into practical implementations using Spring AI. While the pattern descriptions and diagrams are sourced from Anthropic's original publication, we'll focus on how to implement these patterns using Spring AI's features for model portability and structured output. We recommend…
We're excited to introduce Spring AI MCP, a robust Java SDK implementation of the Model Context Protocol (MCP).
This new addition to the Spring AI ecosystem brings standardized AI model integration capabilities to the Java platform. What is MCP? The Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to Large Language Models (LLMs).
MCP provides a standardized way to connect AI models to different data sources and tools, making integration seamless and consistent.
It helps you build agents and complex workflows on top of LLMs. LLMs frequently…
The Amazon Bedrock Nova models represent a new generation of foundation models supporting a broad range of use cases, from text and image understanding to video-to-text analysis. With the Spring AI Bedrock Converse API integration, developers can seamlessly connect to these advanced Nova models and build sophisticated conversational applications with minimal effort. This blog post introduces the key features of Amazon Nova models, demonstrates their integration with Spring AI's Bedrock Converse API, and provides practical examples for text, image, video, document processing, and function…
