Building the POC with LLMs

The short version

You send the model a request over HTTPS and get a response back, just like any other web API. The official Python package, anthropic, wraps this for you. A request has a model id, a maximum number of output tokens, an optional system prompt and a list of messages. The response contains content blocks, usually text, plus a stop reason and token usage.

For a program to use the answer, ask for JSON that follows a schema you define, then validate it in your code. If it does not pass, you reject it or send it for review. Never pass unchecked model output into a database or another system.

Setting up

Install the package with pip install anthropic, and also pydantic for validation. Create an API key in the Claude Console and store it in an environment variable called ANTHROPIC_API_KEY. The client reads it automatically, so the key never appears in your code. On a client project, keys come from their secrets manager, not from a file in the repository.

Choose a model id from the models overview page and keep it in one place in your code or configuration. The example in this lesson uses claude-sonnet-5-5, a current model at the time of writing. Model ids change over time, so check the page rather than copying an id from an old project.

System prompt and messages

The system prompt sets the role, rules and context for the whole conversation: what the task is, what to do when information is missing, and what tone or format to use. User messages carry the specific input for this request. Keep the stable instructions in the system prompt and the changing data in the user message.

Put input data inside clear tags such as <invoice>...</invoice>. This helps the model tell your instructions apart from the content it is processing, and it is also a first defence against instructions hidden in that content. Be explicit about edge cases: 'If a field is missing, return an empty string' is far better than hoping the model guesses what you want.

Treat prompts like code. Keep them in files under version control, change one thing at a time, and rerun your test examples after each change.

Getting JSON in a fixed shape

There are two levels of control. The simple level is asking for JSON in the prompt and parsing the reply. That works most of the time but can fail on edge cases, for example extra text around the JSON. The stronger level is structured outputs: you pass a JSON schema in the output_config parameter, and the API constrains the response to match that schema. Check the structured outputs page for which models support it and which schema features are allowed; the schema in the example sets additionalProperties to false and lists every required field.

The Python SDK also offers client.messages.parse, which takes a Pydantic model directly and returns the parsed object. Both routes are shown in the official structured outputs page. The example below uses the raw schema so you can see exactly what is sent.

import anthropic
from pydantic import BaseModel, ValidationError

class Invoice(BaseModel):
    supplier: str
    invoice_number: str
    total: float
    currency: str

FIELDS = {"supplier": "string", "invoice_number": "string", "total": "number", "currency": "string"}
SCHEMA = {
    "type": "object",
    "properties": {name: {"type": kind} for name, kind in FIELDS.items()},
    "required": list(FIELDS),
    "additionalProperties": False,
}
SYSTEM = ("You extract fields from supplier invoices. Use only facts in the text. "
          "If a field is missing, return an empty string for it.")

client = anthropic.Anthropic()  # reads the ANTHROPIC_API_KEY environment variable
text = "Invoice INV-2291 from Northwind Supplies. Total due: 1,240.50 EUR."
response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    system=SYSTEM,
    messages=[{"role": "user", "content": f"<invoice>{text}</invoice>"}],
    output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
)
if response.stop_reason != "end_turn":
    raise RuntimeError(f"Unexpected stop: {response.stop_reason}")
raw = next(block.text for block in response.content if block.type == "text")
try:
    invoice = Invoice.model_validate_json(raw)
    print(invoice)
except ValidationError as err:
    print("Rejected:", err)

Validating the output

A schema guarantees shape, not truth. The model can still return a total of 0, a supplier name it made up, or a currency code that does not exist. So validate in two steps. First, parse into a typed object, which the code does with Pydantic's model_validate_json. Second, apply business rules: totals must be positive, currency must be in an allowed list, the invoice number must match the client's format, and values should appear in the source text.

Also check the stop reason. end_turn means the model finished normally. max_tokens means it ran out of room and the output may be cut off. refusal means it declined. Handle each one on purpose instead of assuming success. Record failures with the input that caused them; they become test cases for your next prompt change.

From one call to a batch run

A POC usually processes a folder of real items, not one string. Wrap the call in a function that takes one item and returns either a validated object or an error record, then loop over the items and write the results to a CSV or SQLite table with the item id, the output, the stop reason, the validation result and the token usage from the response. That table is your first evaluation report: you can sort by failures, count how many passed, and work out cost per item.

Add basic resilience. The SDK retries some temporary errors for you, but you should still catch exceptions per item so one bad file does not stop the run, and set a sensible timeout. For large runs that do not need instant answers, the batch processing API accepts many requests at once at a lower price and returns the results later.

The short version

You describe one or more tools to the model: a name, a plain-language description and a JSON schema for the inputs. When the model decides a tool would help, it does not run anything. It replies with a request: 'call get_order_status with order_id A-1002'. Your code runs the function, sends the result back, and the model uses it to continue.

This is also called function calling. The key idea is that the model proposes and your code disposes. You stay in control of what actually runs, with what permissions, and what the model gets to see.

Defining a tool

A tool definition has three parts. The name is a short identifier such as get_order_status. The description says what the tool does, when to use it and what the inputs mean. The input_schema is a JSON schema listing the parameters, their types and which are required.

The description matters more than people expect. The model decides whether and how to call a tool mostly from that text. 'Gets order' is weak. 'Look up the delivery status of one customer order by its id, for example A-1001' is clear. Say what the tool does not do as well, if confusion is likely.

Keep tools small and specific. One tool per clear action is easier for the model to use correctly and easier for you to secure and test than one large tool with many modes.

The loop, step by step

The loop is the core of tool use. Get it right once and you will reuse it everywhere.

• Send the user's message plus the tool definitions to the API.
• Read the response. If stop_reason is 'tool_use', it contains one or more tool_use blocks, each with an id, a tool name and an input object.
• Add the assistant's full response to the message history unchanged.
• For each tool_use block, run your matching function with the given input.
• Send a new user message containing a tool_result block for each call, with the matching tool_use_id and the output as content.
• Call the API again with the updated history. Repeat until stop_reason is not 'tool_use', then read the final text.

Handling results and errors

Tool results are plain text or structured content. Return what the model needs to answer, not a full database row with internal fields. If a tool fails, still send a tool_result, mark it with is_error set to true, and describe the problem in words, such as 'no order with that id'. The model can then ask the user for a correct id instead of inventing an answer.

Validate the input the model sends before running anything. The schema helps, and strict tool use can enforce it, but your function should still check values: is this order id in the expected format, and does the current user have the right to see it? The model choosing a tool is not the same as the user being allowed to use it.

The model may ask for several tools in one response. The loop in the example handles that by collecting a result for every tool_use block before calling the API again.

Choosing what to expose

Start with read-only tools. A tool that looks things up cannot do much damage if the model calls it at the wrong time. Add tools that change data only when needed, and put a human approval step in front of them, as covered in the security lesson.

The SDK also provides a tool runner helper that runs this loop for you. It is worth using once you understand the loop. Build it by hand first, so that you know what is happening when something goes wrong.

Controlling when tools are used

By default the model decides on each turn whether to call a tool or answer directly. That is usually what you want. The tool_choice parameter lets you change it: you can require that some tool is used, force one named tool, or turn tools off for a turn. Forcing a tool is handy when the tool is really a way to get structured data out, for example a record_invoice tool whose input schema is the shape you want.

Your system prompt also steers tool use. A line such as 'Always look up the order before answering questions about delivery' makes the model check live data instead of answering from general knowledge. If the model asks for a value the user has not given, such as a missing order id, it may ask the user, or it may guess. Check for guessed inputs in your function, and tell the model in the prompt to ask when a required value is missing.

Every tool definition is sent with every request, so it adds input tokens. Keep the tool list to what the task needs.

The short version

RAG has two phases. Before any question is asked, you prepare the documents: split them into small pieces called chunks and store them in a searchable index. When a question arrives, you search the index for the most relevant chunks, put them in the prompt with the question, and ask the model to answer using only those chunks and to cite them.

An everyday comparison: an open-book exam. The model is the student, the index is the textbook with a good table of contents, and citations are the page numbers that let the examiner check the answer.

Chunking

A chunk is a piece of a document small enough to be relevant to one idea and large enough to make sense alone. Common starting points are a few hundred words, or splitting on headings and paragraphs. Many pipelines add a small overlap between neighbouring chunks so a sentence that crosses a boundary is not lost.

Keep metadata with every chunk: the source file, section heading, page number, date and who is allowed to see it. You need this for citations and for permission filtering. Structure-aware splitting, for example one chunk per contract clause, usually beats splitting at a fixed word count.

Searching: keywords, embeddings or both

Keyword search finds chunks that share words with the question. It is simple, fast, needs no extra service and is excellent for exact terms like product codes and policy names. SQLite's built-in FTS5 extension, used in the code below, ranks results with the BM25 method.

Embedding search turns each chunk and the question into a list of numbers, a vector, that represents meaning. Chunks whose vectors are close to the question's vector are about the same thing even if they use different words, for example 'annual leave' and 'vacation days'. You need an embedding model and a place to store vectors, such as a vector database or a Postgres extension. Anthropic does not offer its own embedding model; its documentation points to providers such as Voyage AI.

Many production systems use both and merge the results. For a first POC, keyword search is often good enough to prove value and much easier to debug.

import sqlite3

def chunk(text: str, size: int = 20, overlap: int = 5) -> list[str]:
    words = text.split()
    step = size - overlap
    return [" ".join(words[i:i + size]) for i in range(0, max(len(words) - overlap, 1), step)]

policy = {
    "travel-policy.md": "Economy class is required for flights under six hours. "
        "Business class needs written approval from a director. Hotel stays are "
        "capped at the city rate published by finance each January.",
    "expenses-faq.md": "Receipts are required for every expense over 25 EUR. "
        "Claims must be submitted within 30 days of the trip end date.",
}
db = sqlite3.connect(":memory:")
db.execute("CREATE VIRTUAL TABLE chunks USING fts5(source, body)")
for source, text in policy.items():
    for piece in chunk(text):
        db.execute("INSERT INTO chunks VALUES (?, ?)", (source, piece))

def search(question: str, k: int = 3) -> list[tuple[str, str]]:
    terms = " OR ".join(w.strip("?.,").lower() for w in question.split() if len(w) > 3)
    sql = "SELECT source, body FROM chunks WHERE chunks MATCH ? ORDER BY rank LIMIT ?"
    return db.execute(sql, (terms, k)).fetchall()

for source, body in search("Do I need approval for business class flights?"):
    print(source, "|", body[:70])

Putting retrieved text in the prompt

Place each retrieved chunk in its own tagged block with an id and its source. Tell the model to answer only from the sources, to cite the id after each claim, and to say it does not know when the sources do not contain the answer. That last instruction is what stops the model filling gaps with plausible guesses.

Send a handful of good chunks rather than many weak ones. Too much loosely related text can distract the model and costs more. The Claude API also has a citations feature that returns exact cited passages from documents you pass in; it is worth reading about once your basic pipeline works.

Checking quality

Test retrieval and generation separately. For retrieval, write 20 to 50 real questions with the chunk that should answer each one, and measure how often that chunk appears in the top results. For generation, check that answers are correct, that citations point to chunks that actually support the claim, and that the system says 'I do not know' when it should.

When answers are wrong, look at what was retrieved first. Most RAG problems are retrieval problems: the right chunk was never found, was split in half, or was ranked below irrelevant ones.

Keeping the index fresh and safe

Documents change. Policies get a new version, contracts are amended, old manuals are withdrawn. Plan how the index is updated: a nightly job that re-reads changed files, removes chunks from deleted files and records the document version on each chunk. Show the document date next to each citation so users can see if an answer comes from an old version.

Retrieved text is untrusted input, just like an email. A document in the index could contain hidden instructions, so the security controls from the previous module apply here too: mark sources as data, keep tools limited and check outputs. Also apply each user's document permissions at search time. A RAG system that returns a confidential board paper to anyone who asks the right question is a data leak with a friendly interface.

Finally, keep the original documents reachable from the citation, so a user can open the source and read the full context in one click.

The short version

A workflow is a path you design in code: step one, then step two, maybe a branch. The model fills in parts, such as classifying or drafting, but your code decides the order. An agent is a model that decides its own next step in a loop: it looks at the goal and what has happened so far, picks a tool, sees the result, and repeats until it thinks the job is done.

Anthropic's guide 'Building effective agents' makes this same distinction and recommends starting with the simplest approach that works, adding agent behaviour only when it clearly helps. Agents trade cost, speed and predictability for flexibility.

Workflows first

Several workflow patterns cover most business tasks. Prompt chaining splits a job into fixed steps, such as extract, then check, then draft. Routing classifies the input and sends it down the right path. Parallel calls run several checks at once and combine them. An evaluator step asks a second call to review the first one's output.

If you can draw the steps on a whiteboard in advance, build a workflow. It is easier to test because each step has a clear input and output, and it is easier to explain to the client and their auditors.

When an agent fits

Use an agent when the number and order of steps truly cannot be known in advance and depends on what is found along the way. Examples: investigating why a shipment is late across several systems, researching a question across many documents, or fixing a failing test in a codebase. Coding assistants are a good example of agents that work well, because the environment gives clear feedback: the tests pass or fail.

Good agent tasks also have a clear finish line and a way to check the result. Without those, an agent can wander, spend a lot and still not be clearly right.

Weigh the costs honestly. Each turn of an agent is another model call, so an agent task may cost many times more than a single call and take much longer. Its path also changes from run to run, which makes testing harder: you test outcomes on a set of cases rather than checking each step. Explain these trade-offs to the client before you recommend one.

A useful middle ground is a workflow with one agent step inside it. The fixed parts, such as fetching the case and writing the result, stay predictable, and only the investigation in the middle is left to the agent, with its own turn limit.

The agent loop

Underneath, an agent is the tool-use loop from the previous lesson, run with more tools and less fixed structure. The model receives the goal and the tools. It thinks about the next step, asks for a tool, your code runs it, the result goes back, and the model decides again. The loop ends when the model gives a final answer, when it asks for a person, or when a limit is hit.

Context grows with every turn, because each tool call and result is added to the history. Long-running agents need care with what goes into context: keep tool results short and summarize old steps. Anthropic's article on context engineering covers this in depth.

The system prompt for an agent should describe the goal, the tools and when to use each, what a finished result looks like, and when to stop and ask a person. Good tool results help too: a clear message such as 'no order with that id, check the format' lets the agent correct itself, while a raw stack trace often sends it in circles. You do not need a framework to build this loop; the code below is plain Python on top of the API, which keeps every step visible.

Guardrails

Guardrails are the rules around the loop that keep the agent safe even when the model makes a poor choice. Put them in code, not only in the prompt.

Many teams also add an evaluator or a person at the end to check the final result before it is used.

• A maximum number of turns and a maximum cost per task.
• A small set of tools, read-only by default.
• Human approval for any action that sends, pays, deletes or changes access.
• Clear error results so the agent can recover instead of looping.
• An audit log of every tool call, input and output.
• A defined hand-over to a person when the agent is stuck.

import anthropic

MAX_TURNS = 8                       # hard stop so a confused agent cannot loop forever
NEEDS_APPROVAL = {"issue_refund"}   # actions a person must confirm

def run_tool(name: str, args: dict) -> tuple[str, bool]:
    if name == "lookup_order":
        return f"Order {args['order_id']}: paid 49.00, delivered damaged", False
    if name == "issue_refund":
        answer = input(f"Approve refund {args}? (yes/no) ")
        if answer.strip().lower() != "yes":
            return "A person declined this refund. Do not retry.", True
        return f"Refund of {args['amount']} issued", False
    return f"Unknown tool {name}", True

def run_agent(client, task: str, tools: list) -> str:
    messages = [{"role": "user", "content": task}]
    for turn in range(MAX_TURNS):
        response = client.messages.create(
            model="claude-sonnet-5-5", max_tokens=2048, tools=tools, messages=messages)
        if response.stop_reason != "tool_use":
            return next((b.text for b in response.content if b.type == "text"), "")
        messages.append({"role": "assistant", "content": response.content})
        results = []
        for block in response.content:
            if block.type == "tool_use":
                output, failed = run_tool(block.name, block.input)
                print(f"turn {turn}: {block.name} -> {output}")   # audit trail
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": output, "is_error": failed})
        messages.append({"role": "user", "content": results})
    return "Stopped: turn limit reached, handing over to a person."

The short version

MCP is an open standard for connecting AI applications to data and tools. Think of it like a standard plug: any device with the right socket can use any appliance with the right plug. An MCP server exposes capabilities, such as tools, and an MCP client inside an AI application connects to it and makes those capabilities available to the model.

Anthropic introduced MCP and released it as an open protocol, and it is now supported by many AI applications and developer tools. The official site, modelcontextprotocol.io, holds the specification, guides and SDKs for several languages.

Hosts, clients and servers

The host is the AI application the user works in, for example Claude Desktop, Claude Code or your own app. Inside the host, an MCP client manages the connection to one MCP server. The server is a separate program that provides the capabilities, such as access to a database, a file system or a company API.

Servers can run locally, started by the host and talking over standard input and output (the stdio transport), or remotely, reached over HTTP. Local servers are simple for development and personal tools. Remote servers suit shared company connectors, where authentication and access control matter.

Tools, resources and prompts

A server can offer three main kinds of capability. Tools are functions the model can call, like the tools in the previous lesson, for example lookup_policy. Resources are data the application can read and put in context, identified by a URI, for example a list of policy topics or a file. Prompts are reusable templates a user can pick, for example 'check this expense claim against policy'.

The difference is about who is in control. The model decides when to call tools. The application or user decides which resources to include. The user chooses prompts explicitly. Most servers start with tools only and add resources and prompts when they help.

A minimal server in Python

The official Python SDK is installed with pip install "mcp[cli]" (the official tutorial uses uv add "mcp[cli]") and needs Python 3.10 or newer. The current tutorial creates a server with the MCPServer class and registers capabilities with decorators. Type hints and the docstring become the tool's input schema and description, so write them carefully. Older tutorials use a class called FastMCP; check the version of the guide you are following.

The example below defines one tool, one resource and one prompt, then runs over stdio. Because a stdio server talks through standard input and output, do not print debug messages to standard output from inside it; use logging to standard error instead, as the official guide explains.

from mcp.server import MCPServer

mcp = MCPServer("policy-helper")

POLICIES = {
    "travel": "Economy class for flights under six hours. Business class needs director approval.",
    "expenses": "Receipts are required for every expense over 25 EUR.",
}

@mcp.tool()
def lookup_policy(topic: str) -> str:
    """Return the company policy text for a topic.

    Args:
        topic: One of travel or expenses
    """
    return POLICIES.get(topic.lower(), f"No policy found for {topic}")

@mcp.resource("policy://index")
def policy_index() -> str:
    """List the policy topics this server knows about."""
    return ", ".join(POLICIES)

@mcp.prompt()
def check_claim(claim: str) -> str:
    """Ask the model to check an expense claim against policy."""
    return f"Check this expense claim against company policy and list any problems:\n{claim}"

if __name__ == "__main__":
    mcp.run(transport="stdio")

Testing and connecting

Test a server before connecting it to any assistant. The MCP Inspector is the official developer tool: run npx @modelcontextprotocol/inspector followed by the command that starts your server, for example python server.py. It opens a browser view where you can list tools, call them with test inputs and see the raw messages. It needs a recent version of Node.js.

To use the server from Claude Code, register it with claude mcp add, for example claude mcp add --transport stdio policy-helper -- python /full/path/server.py, then check it with claude mcp list. Other hosts have their own configuration steps, described in their documentation.

Treat every MCP server like any other software you install. A server can read data and take actions with the permissions you give it. Use servers from trusted sources, read what each tool does, and give remote servers only the access they need.

When MCP is worth it

MCP adds a separate program and a protocol between your app and your tools. That is worth it when the same capability will be used from more than one host, when the client wants their staff to use company data inside off-the-shelf assistants, or when a ready-made MCP server already exists for the system you need, such as a database, a ticket tracker or a file store. The official servers repository and many vendors publish servers you can start from.

If you are building one custom application with two or three tools that nobody else will use, plain tool use inside the app is simpler and has fewer moving parts. You can still move those functions into an MCP server later, because the idea is the same: a name, a description, an input schema and a function.

For a shared, remote server, plan authentication from the start. Each user should reach only the data their own account allows, and every call should be logged.

The short version

Modern coding assistants do more than complete lines. In agent modes they read files, run commands and tests, and edit many files to complete a task. Claude Code works in the terminal, IDEs and desktop app; Cursor is an editor with built-in agents; GitHub Copilot works in editors and on GitHub. The ideas in this lesson apply to all of them.

Treat the assistant like a very fast, capable new colleague who does not know the project. You set the goal, give context, agree a plan, let it build, make it prove the result with tests, and review the change yourself.

The loop: plan, build, test, review

Plan. Ask the assistant to read the relevant code and propose a plan before writing anything. Claude Code has a plan mode for exactly this. Read the plan, correct it, and only then let it build. Skip planning for tiny, obvious changes.

Build. Let it implement one small piece at a time. Small steps are easier to check and easier to undo. Commit working states to Git often, so you can always return to the last good version.

Test. Give the assistant a check it can run itself: a test suite, a build, a script that compares output to an expected file. The Claude Code best practices guide calls this the single most useful habit, because the assistant can then loop until the check passes instead of stopping when the code merely looks done.

Review. Read the diff yourself. Look for changes outside the task, deleted tests, hard-coded values, secrets and missing error handling. A fresh review session or a second assistant asked to find gaps can help, but the decision to merge is yours.

How to brief the assistant

Most poor results come from vague requests. A good brief states the goal, points to the files and existing patterns, lists constraints, gives example inputs and outputs, says what 'done' means in a way that can be checked, and names what is out of scope. The worked example shows one.

Put long-lived project knowledge in a project memory file so you do not repeat it every time. Claude Code reads CLAUDE.md, Cursor uses rules files and Copilot supports custom instructions. Keep these short: the commands to build and test, code style rules that differ from defaults, and project-specific gotchas. Long files get ignored.

Keeping control of quality

Write or agree the tests first when you can. Tests written before the code describe what you want, independently of how the assistant chose to build it. The test file in this section was written before asking for the function; the assistant's job is to make it pass without changing it.

Watch for an assistant 'fixing' a failing test by weakening it or deleting it. Say in the brief that tests must not be changed without asking. Keep permission settings sensible: let the assistant run tests and read files freely, but approve commands that touch remote systems, delete data or deploy. Never paste client secrets into a prompt.

Know the client's rules on AI coding tools. Some clients restrict which tools may see their code. Ask in the first week, not after you have used one.

# Written before asking the assistant to build parse_amount. Run with: pytest -q
import pytest
from invoice_utils import parse_amount

def test_plain_amount():
    assert parse_amount("1240.50 EUR") == (1240.50, "EUR")

def test_thousands_separator():
    assert parse_amount("1,240.50 EUR") == (1240.50, "EUR")

def test_dollar_symbol():
    assert parse_amount("$99") == (99.0, "USD")

def test_rejects_text_without_amount():
    with pytest.raises(ValueError):
        parse_amount("see attached")

Managing a long session

Assistants work within a context window that fills up with files, command output and conversation. As it fills, quality drops. Start a fresh session between unrelated tasks, keep investigations scoped, and if you have corrected the same mistake twice, start again with a better brief that includes what you learned. Use subagents or a separate session for broad research so the main session stays focused.

Using the assistant to understand a codebase

On a new engagement, one of the most valuable uses of a coding assistant is reading, not writing. Ask it the questions you would ask a senior engineer on the client team: how does authentication work here, where are environment variables loaded, how is a new API endpoint added, what does this job do every night. Ask it to point to the files and lines behind each answer, then open them and check. You will learn the codebase in hours instead of days.

Use what you learn to write better briefs. A brief that says 'follow the pattern in the existing orders endpoint' produces code that fits the client's style, which matters at handover. The client's developers will maintain this code after you leave, so it should look like their code, use their libraries and pass their checks, not look like it came from somewhere else.