The Comprehensive AI Agents Guide: What They Are, How They Work, and How to Build Them

Agent Design Foundations

In its most fundamental form, an agent consists of three core components, according to OpenAI's practical guide:

• Model: The LLM powering the agent's reasoning and decision-making.
• Tools: External functions or APIs the agent can use to take action.
• Instructions: Explicit guidelines and guardrails defining how the agent behaves.

Here is what this looks like in code when using OpenAI's Agents SDK. You can also implement the same concepts using your preferred library or building directly from scratch.

weather_agent = Agent(
    name="Weather agent",
    instructions="You are a helpful agent who can talk to users about the weather",
    tools=[get_weather],
)

Selecting Your Models

Different models have different strengths and tradeoffs related to task complexity, latency, and cost. As we will see in the next section on Orchestration, you might want to consider using a variety of models for different tasks in the workflow. Not every task requires the smartest model. A simple retrieval or intent classification task may be handled by a smaller, faster model, while harder tasks like deciding whether to approve a refund may benefit from a more capable model.

An approach that works well is to build your agent prototype with the most capable model for every task to establish a performance baseline. From there, try swapping in smaller models to see if they still achieve acceptable results. This way, you don't prematurely limit the agent's abilities, and you can diagnose where smaller models succeed or fail.

In summary, the principles for choosing a model are simple:

• Set up evals to establish a performance baseline.
• Focus on meeting your accuracy target with the best models available.
• Optimize for cost and latency by replacing larger models with smaller ones where possible.

Defining Tools

Tools extend your agent's capabilities by using APIs from underlying applications or systems. For legacy systems without APIs, agents can rely on computer-use models to interact directly with those applications and systems through web and application UIs, just as a human would.

Each tool should have a standardized definition, enabling flexible, many-to-many relationships between tools and agents. Well-documented, thoroughly tested, and reusable tools improve discoverability, simplify version management, and prevent redundant definitions.

Broadly speaking, agents need three types of tools:

Type	Description	Examples
Data	Enable agents to retrieve context and information necessary for executing the workflow.	Query transaction databases or systems like CRMs, read PDF documents, or search the web.
Action	Enable agents to interact with systems to take actions such as adding new information to databases, updating records, or sending messages.	Send emails and texts, update a CRM record, hand-off a customer service ticket to a human.
Orchestration	Agents themselves can serve as tools for other agents.	Refund agent, Research agent, Writing agent.

For example, here is how you would equip the agent defined above with a series of tools when using the Agents SDK:

from agents import Agent, WebSearchTool, function_tool
import datetime

@function_tool
def save_results(output):
    db.insert({
        "output": output,
        "timestamp": datetime.datetime.now(),
    })
    return "File saved"

search_agent = Agent(
    name="Search agent",
    instructions="Help the user search the internet and save results if asked.",
    tools=[WebSearchTool(), save_results],
)

As the number of required tools increases, consider splitting tasks across multiple agents.

Configuring Instructions

High-quality instructions are essential for any LLM-powered app, but especially critical for agents. Clear instructions reduce ambiguity and improve agent decision-making, resulting in smoother workflow execution and fewer errors.

Best Practices for Agent Instructions

• Use existing documents: When creating routines, use existing operating procedures, support scripts, or policy documents to create LLM-friendly routines. In customer service for example, routines can roughly map to individual articles in your knowledge base.
• Prompt agents to break down tasks: Providing smaller, clearer steps from dense resources helps minimize ambiguity and helps the model better follow instructions.
• Define clear actions: Make sure every step in your routine corresponds to a specific action or output. For example, a step might instruct the agent to ask the user for their order number or to call an API to retrieve account details. Being explicit about the action (and even the wording of a user-facing message) leaves less room for errors in interpretation.
• Capture edge cases: Real-world interactions often create decision points such as how to proceed when a user provides incomplete information or asks an unexpected question. A robust routine anticipates common variations and includes instructions on how to handle them with conditional steps or branches such as an alternative step if a required piece of info is missing.

You can use advanced models, like o1 or o3-mini, to automatically generate instructions from existing documents. Here is a sample prompt illustrating this approach:

"You are an expert in writing instructions for an LLM agent.
Convert the following help center document into a clear set of instructions,
written in a numbered list.
The document will be a policy followed by an LLM.
Ensure that there is no ambiguity, and that the instructions are written as directions for an agent.
The help center document to convert is the following {{help_center_doc}}"

Orchestration Patterns

With the foundational components in place, you can consider orchestration patterns to enable your agent to execute workflows effectively. While it is tempting to immediately build a fully autonomous agent with complex architecture, customers typically achieve greater success with an incremental approach.

In general, orchestration patterns fall into two categories:

• Single-agent systems, where a single model equipped with appropriate tools and instructions executes workflows in a loop.
• Multi-agent systems, where workflow execution is distributed across multiple coordinated agents.

Single-agent Systems

A single agent can handle many tasks by incrementally adding tools, keeping complexity manageable and simplifying evaluation and maintenance. Each new tool expands its capabilities without prematurely forcing you to orchestrate multiple agents.

Every orchestration approach needs the concept of a 'run', typically implemented as a loop that lets agents operate until an exit condition is reached. Common exit conditions include tool calls, a certain structured output, errors, or reaching a maximum number of turns.

For example, in the Agents SDK, agents are started using the method, which loops over the LLM until either:

• A final-output tool is invoked, defined by a specific output type.
• The model returns a response without any tool calls (e.g., a direct user message).

Example usage:

Agents.run(
    agent,
    [UserMessage("What's the capital of the USA")]
)

This concept of a while loop is central to the functioning of an agent. In multi-agent systems, as you will see next, you can have a sequence of tool calls and handoffs between agents but allow the model to run multiple steps until an exit condition is met.

An effective strategy for managing complexity without switching to a multi-agent framework is to use prompt templates. Rather than maintaining numerous individual prompts for distinct use cases, use a single flexible base prompt that accepts policy variables. This template approach adapts easily to various contexts, significantly simplifying maintenance and evaluation. As new use cases arise, you can update variables rather than rewriting entire workflows.

""" You are a call center agent. You are interacting with
{{user_first_name}} who has been a member for {{user_tenure}}. The user's
most common complains are about {{user_complaint_categories}}. Greet the
user, thank them for being a loyal customer, and answer any questions the
user may have!

When to Consider Creating Multiple Agents

Our general recommendation is to maximize a single agent's capabilities first. More agents can provide intuitive separation of concepts, but can introduce additional complexity and overhead, so often a single agent with tools is sufficient. For many complex workflows, splitting up prompts and tools across multiple agents allows for improved performance and scalability. When your agents fail to follow complicated instructions or consistently select incorrect tools, you may need to further divide your system and introduce more distinct agents.

Practical guidelines for splitting agents include:

• Complex logic: When prompts contain many conditional statements (multiple if-then-else branches), and prompt templates get difficult to scale, consider dividing each logical segment across separate agents.
• Tool overload: The issue is not solely the number of tools, but their similarity or overlap. Some implementations successfully manage more than 15 well-defined, distinct tools while others struggle with fewer than 10 overlapping tools.

Multi-agent Systems

A multiagent system consists of multiple AI agents working collectively to perform tasks on behalf of a user or another system. Individual AI agents can be specialized to perform specific subtasks with accuracy. An orchestrator agent coordinates the activities of different specialist agents to complete larger, more complex tasks.

Workflow Patterns

Anthropic's engineering team has identified several common workflow patterns for agentic systems:

Prompt Chaining

Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks on any intermediate steps to ensure that the process is still on track.

When to use this workflow: This workflow is ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks. The main goal is to trade off latency for higher accuracy, by making each LLM call an easier task.

Examples where prompt chaining is useful:

• Generating Marketing copy, then translating it into a different language.
• Writing an outline of a document, checking that the outline meets certain criteria, then writing the document based on the outline.

Routing

Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs.

When to use this workflow: Routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model/algorithm.

Examples where routing is useful:

• Directing different types of customer service queries (general questions, refund requests, technical support) into different downstream processes, prompts, and tools.
• Routing easy/common questions to smaller, cost-efficient models like Claude Haiku 4.5 and hard/unusual questions to more capable models like Claude Sonnet 4.5 to optimize for best performance.

Parallelization

LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically. This workflow, parallelization, manifests in two key variations:

• Sectioning: Breaking a task into independent subtasks run in parallel.
• Voting: Running the same task multiple times to get diverse outputs.

When to use this workflow: Parallelization is effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results. For complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect.

Examples where parallelization is useful:

• Sectioning:
  • Implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.
  • Automating evals for evaluating LLM performance, where each LLM call evaluates a different aspect of the model's performance on a given prompt.
• Voting:
  • Reviewing a piece of code for vulnerabilities, where several different prompts review and flag the code if they find a problem.
  • Evaluating whether a given piece of content is inappropriate, with multiple prompts evaluating different aspects or requiring different vote thresholds to balance false positives and negatives.

Orchestrator-workers

In the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.

When to use this workflow: This workflow is well-suited for complex tasks where you cannot predict the subtasks needed (in coding, for example, the number of files that need to be changed and the nature of the change in each file likely depend on the task). Whereas it is topographically similar, the key difference from parallelization is its flexibility. Subtasks are not pre-defined, but determined by the orchestrator based on the specific input.

Example where orchestrator-workers is useful:

• Coding products that make complex changes to multiple files each time.
• Search tasks that involve gathering and analyzing information from multiple sources for possible relevant information.

Evaluator-optimizer

In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop.

When to use this workflow: This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative writing process a human writer might go through when producing a polished document.

Examples where evaluator-optimizer is useful:

• Literary translation where there are nuances that the translator LLM might not capture initially, but where an evaluator LLM can provide useful critiques.
• Complex search tasks that require multiple rounds of searching and analysis to gather comprehensive information, where the evaluator decides whether further searches are warranted.

Agents (Autonomous)

Agents are emerging in production as LLMs mature in key capabilities. Understanding complex inputs, engaging in reasoning and planning, using tools reliably, and recovering from errors. Agents begin their work with either a command from, or interactive discussion with, the human user. Once the task is clear, agents plan and operate independently, potentially returning to the human for further information or judgement. During execution, it is crucial for the agents to gain "ground truth" from the environment at each step (such as tool call results or code execution) to assess its progress. Agents can then pause for human feedback at checkpoints or when encountering blockers. The task often terminates upon completion, but it is also common to include stopping conditions (such as a maximum number of iterations) to maintain control.

Agents can handle sophisticated tasks, but their implementation is often straightforward. They are typically just LLMs using tools based on environmental feedback in a loop. It is therefore crucial to design toolsets and their documentation clearly and thoughtfully.

When to use agents: Agents can be used for open-ended problems where it is difficult or impossible to predict the required number of steps, and where you cannot hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents' autonomy makes them ideal for scaling tasks in trusted environments. The autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails.

Examples where agents are useful:

• A coding Agent to resolve SWE-bench tasks, which involve edits to many files based on a task description.
• Anthropic's "computer use" reference implementation, where Claude uses a computer to accomplish tasks.

Building Agents with Hugging Face

Hugging Face provides tools and protocols that connect AI agents directly to the Hub. Whether you are chatting with Claude, building with Codex, or developing custom agents, you can access models, datasets, Spaces, and community tools.

Chat with Hugging Face

Connect your AI assistant directly to the Hugging Face Hub using the Model Context Protocol (MCP). Once connected, you can search models, explore datasets, generate images, and use community tools, all from within your chat interface.

Supported Assistants

The HF MCP Server works with any MCP-compatible client:

• ChatGPT (via plugins)
• Claude Desktop
• Custom MCP clients

Setup

1. Open MCP Settings at huggingface.co/settings/mcp while logged in.
2. Select your MCP-compatible client from the list. The page shows client-specific instructions and a ready-to-copy configuration snippet.
3. Copy the configuration snippet into your client's MCP settings, save, and restart your client.

The settings page generates the exact configuration your client expects. Use it rather than writing config by hand.

What You Can Do

Once connected, ask your assistant to use Hugging Face tools among the ones you selected in your configuration:

Task	Example Prompt
Search models	"Find Qwen 3 quantizations on Hugging Face"
Explore datasets	"Show datasets about weather time-series"
Find Spaces	"Find a Space that can transcribe audio files"
Generate images	"Create a 1024x1024 image of a cat in Ghibli style"
Search papers	"Find recent papers on vision-language models"

Your assistant calls MCP tools exposed by the Hugging Face server and returns results with metadata, links, and context.

Coding Agents

Integrate Hugging Face into your coding workflow with the MCP Server and Skills. Access models, datasets, and ML tools directly from your IDE or coding agent.

Quick Setup

MCP Server: The MCP Server gives your coding agent access to Hub search, Spaces, and community tools.

For Cursor / VS Code / Zed:

• Visit huggingface.co/settings/mcp
• Select your IDE from the list
• Copy the configuration snippet
• Add it to your IDE's MCP settings
• Restart the IDE

For Claude Code:

claude mcp add hf-mcp-server -t http "https://huggingface.co/mcp?login"

Skills: Skills provide task-specific guidance for AI/ML workflows. They work alongside MCP or standalone.

# start claude
claude

# install the skills marketplace plugin
/plugin marketplace add huggingface/skills

Then, to install a Skill specification:

/plugin install hf-cli@huggingface/skills

Environment Configuration

Authentication: Set your Hugging Face token as an environment variable:

export HF_TOKEN="hf_..."

Or authenticate via the CLI:

hf auth login

Example Workflow

You: Find a text classification model that works well on short texts

Agent: [Searches Hugging Face Hub]
Found several options:
- distilbert-base-uncased-finetuned-sst-2-english (sentiment)
- facebook/bart-large-mnli (zero-shot)
...

You: Show me how to use the first one

Agent: [Fetches documentation]
Here's how to use it with transformers:

from transformers import pipeline
classifier = pipeline("sentiment-analysis",
model="distilbert-base-uncased-finetuned-sst-2-english")
result = classifier("I love this product!")

Using Frameworks

There are many frameworks that make agentic systems easier to implement, including:

• The Claude Agent SDK
• Strands Agents SDK by AWS
• Rivet, a drag and drop GUI LLM workflow builder
• Vellum, another GUI tool for building and testing complex workflows

These frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug. They can also make it tempting to add complexity when a simpler setup would suffice.

We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what is under the hood are a common source of customer error.

Agentic RAG

Agentic RAG is the use of AI agents to facilitate retrieval augmented generation (RAG). Compared to traditional RAG systems, agentic RAG allows large language models to conduct information retrieval from multiple sources and handle more complex workflows.

Governance

AI agent governance refers to the processes, standards and guardrails that help ensure agentic systems are safe and ethical.

Benefits of Using AI Agents

According to the AWS documentation, AI agents can improve your business operations and your customers' experiences:

• Improved productivity: Business teams are more productive when they delegate repetitive tasks to AI agents. This way, they can divert their attention to mission-critical or creative activities, adding more value to their organization.
• Reduced costs: Businesses can utilize intelligent agents to minimize unnecessary costs resulting from process inefficiencies, human errors, and manual processes. They can confidently tackle complex tasks because autonomous agents follow a consistent model that adapts to changing environments.
• Informed decision-making: Advanced intelligent agents have predictive capabilities and can collect and process massive amounts of real-time data. This enables business managers to make more informed predictions at speed when strategizing their next move.
• Improved customer experience: Customers seek engaging and personalized experiences when interacting with businesses. Integrating AI agents allows businesses to personalize product recommendations, provide prompt responses, and innovate to improve customer engagement, conversion, and loyalty.

Troubleshooting

Based on community and official sources, here are common failure modes and how to address them:

• Agents fail to follow complicated instructions: If your agent consistently selects incorrect tools or fails to follow instructions, you may need to split your system into more distinct agents. Complex logic with many conditional statements is a sign that you need multiple agents.
• Tool overload: When agents have too many tools, especially overlapping ones, they may struggle to select the correct tool. Keep tools well-defined, distinct, and thoroughly documented. Consider splitting tools across multiple agents if the number exceeds 10-15.
• Compounding errors: Autonomous agents can make errors that compound over multiple turns. Mitigate this by extensive testing in sandboxed environments, using guardrails, and including stopping conditions (e.g., maximum iterations).
• High costs and latency: Agentic systems often trade latency and cost for better task performance. Use smaller, faster models for simpler subtasks and route complex tasks to more capable models. The ReWOO paradigm can reduce token usage by planning upfront.
• Obscured prompts and responses: Frameworks can create abstraction layers that make debugging difficult. Start with direct LLM API calls and only add frameworks when necessary. If you use a framework, ensure you understand the underlying code.

Going Further

After mastering the fundamentals, explore these advanced topics drawn from the source material:

• Multiagent systems: Learn how multiple AI agents can work collectively, with an orchestrator agent coordinating specialist agents.
• Agentic RAG: Combine agents with retrieval augmented generation to handle complex information retrieval workflows.
• Governance: Study the processes, standards, and guardrails for safe and ethical agentic systems.
• Frameworks: Experiment with the Claude Agent SDK, Strands Agents SDK by AWS, Rivet, or Vellum for building and testing complex workflows.
• Hugging Face integration: Connect your agents to the Hugging Face Hub using MCP and Skills to access models, datasets, and community tools.
• Real-world use cases: Explore applications in customer service, human resources, sales, procurement, software design, IT automation, code generation, and conversational assistance.
• The Hugging Face Agents Course: Take the full course at huggingface.co/learn/agents-course to earn certification and build practical skills with smolagents.
• Anthropic's cookbook: Review sample implementations for common patterns like prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer.