Agent Bricks: Automation and Optimization of AI Agents in Databricks

The development of artificial intelligence agents is often a complex process. It involves integrating models, defining training pipelines, optimizing hyperparameters, and ensuring security and governance mechanisms. All this requires considerable effort that often delays the adoption of solutions in real business scenarios.

To overcome these barriers, Databricks introduced Agent Bricks as part of Mosaic AI, a proposal that aims to democratize the construction of AI agents. This service simplifies implementation and optimization, allowing teams to focus on what truly matters: business problems, data, and performance metrics.

What is Agent Bricks?

Agent Bricks is a Databricks service designed to create and optimize high-quality, purpose-specific AI agent systems. Its main value lies in offering a modular approach that abstracts technical complexity and enables production-ready agents without manually assembling each component.

This service leverages Mosaic AI and the Lakehouse Platform, integrating governance, security, vector data, and quality evaluation capabilities. Additionally, being deployed on Azure Geographies ensures compliance with data residency and protection regulations.

Added Value of Agent Bricks

Beyond simplifying agent development, Agent Bricks brings a set of capabilities that make a difference in enterprise environments:

• Automatic evaluations tailored to each case: Instead of relying on generic metrics, the platform generates specific benchmarks for each task, even creating synthetic data or custom LLM judges. This ensures quality is measured with criteria relevant to the business.
• Intelligent optimization of costs and quality: Through techniques like prompt engineering, fine-tuning, reward models, and adaptive strategies (Test-Adaptive Optimization, TAO), the system finds the ideal balance between performance and operational expense.
• Proven efficiency: In document extraction and structured text comprehension tests, Agent Bricks has outperformed solutions that rely solely on prompts, reducing costs by up to 10 times without compromising accuracy.
• More effective human feedback (ALHF): Domain experts can give instructions in natural language (“ignore everything before May 1990”), and the platform translates that feedback into automatic technical adjustments: prompt improvements, filters, patterns, or retrieval schemes.
• Validated impact in production: Organizations like Flo Health, AstraZeneca, and Experian already use Agent Bricks in critical applications such as clinical information extraction, knowledge assistants, and internal agents, reporting higher accuracy, compliance with standards, and accelerated development cycles.

Requirements

To enable Agent Bricks, the workspace must have:

• Preview of Mosaic AI Agent Bricks (Beta) enabled.
• Serverless compute active.
• Unity Catalog enabled.
• Access to foundation models in system.ai.
• Serverless budget policy with a non-zero value.
• Located in a supported region: centralus, eastus, eastus2, northcentralus, southcentralus, westus, westus2.

How does Agent Bricks work?

The workflow proposed by Databricks works as follows:

1. Declare the task: Define in natural language what the agent’s objective is, what it should do, and connect the data sources.
2. Automatic evaluation: The platform will generate benchmarks, build judges if necessary, and produce synthetic data to measure concrete performance.
3. Automatic optimization: The internal engine combines different techniques (prompt engineering, fine-tuning, reward models, adaptive strategies) to maximize quality or minimize costs according to user preference or policy.
4. Choice between cost vs. quality: Users can indicate if they want to prioritize optimal performance or prefer a lower-cost solution that still meets a quality threshold.
5. Continuous improvement: Incorporate human feedback, learning with feedback, automatic updates, refinement of models and technical components.

Supported Use Cases

Agent Bricks currently supports the following use cases, with more to be added soon:

Agent Bricks	Use Case
Information Extraction	Transform untagged text documents into structured tables; extract data points.
Custom LLM	Tasks such as summarization, classification, text transformation.
Knowledge Assistant	Create conversational bots that answer questions and cite sources.
Multi-agent Supervisor	Coordinate multiple agents, integrate agents from different spaces and domains (multi-agent).

Additionally, real-world examples have shown:

In clinical document analysis, Flo Health doubled medical accuracy compared to standard commercial LLMs, maintaining security and privacy standards.

In document parsing tasks, Agent Bricks achieved superior quality compared to models that only optimized their prompts, at a much lower cost.

Conclusion

Agent Bricks consolidates a paradigm shift in building AI agents by offering a modular approach that integrates quality, security, governance, and cost optimization. With this foundation, organizations can accelerate the adoption of reliable artificial intelligence, reducing technical complexity and focusing on what matters: turning data into strategic decisions that drive business.

Beyond its technical potential, Agent Bricks opens the door to an ecosystem where AI innovation is more accessible, scalable, and aligned with business security and compliance requirements.

Resources

• Knowledge Assistant in Agent Bricks
• Key Information Extraction with Agent Bricks (Microsoft Learn)