Module 10 of 11 / 4 lessons / 4 practice exercises / 24 links

Product ownership, demo and hand-over

Building something that works is only half of the job. This module covers how to show it honestly, how to move it from proof of concept to something people rely on, how to plan what comes next in terms of outcomes, and how to hand it to an owner who can run it after you leave.

New to a word? Check Start here: the basics first. How to use this module. Read each lesson top to bottom: the diagram first, then the explanation and the worked example. Use "Find more here" to go deeper with docs, videos and courses. Then do the practice. Tick "I did this" to track your progress in this browser.

10.1Demo craft: show the problem, the result and the limits
10.2From POC to pilot to production
10.3Roadmap, backlog and success measures
10.4The hand-over pack: runbook, owner and support model

Lesson 10.1

Demo craft: show the problem, the result and the limits

You will be able to plan and run a short demo that shows the client's problem, the result on their own data, and the honest limits, and ends in a decision.

Why this matters on a client engagement

The demo is often the moment a sponsor decides whether the work continues. A good demo turns weeks of discovery and building into a clear decision. A bad one, too long, too technical, or hiding the weak spots, can sink good work, or worse, win approval for something that later disappoints. FDEs demo often, so this is a skill worth practising.

Shape of a decision-focused demo

The problemOne real case, in their words
The resultSame case, run live on their data
The evidenceEval set and pass rate by case type
The limitsWhat it does not do; where people decide
The askOne decision and a named owner

Read left to right: each part answers one question the audience has.

The short version

A demo is not a tour of features. It is a short story with four parts: the problem in the client's words, the result running on their real cases, the evidence that it works beyond the cases you show, and the limits you know about. It ends with a clear ask, usually a decision about the next step.

Plan it like a meeting with a purpose, not like a presentation. Before you build a single slide, write down the one decision you want at the end.

A demo script structure

A script keeps the demo short and stops you wandering into detail. A common structure:

Open with the problem: one real case from the client's work, with the time or pain it causes today, in their words.
Show the result: run that same case live, then two or three more that show normal, tricky and refused cases.
Show the evidence: the eval set size, the pass rate by case type, and one failure you understand.
State the limits: what it does not do, what it has not been tested on, and where a person stays in control.
Make the ask: the decision you need, who owns it on the client side, and the next step.

Know your audience

Different people listen for different things. A sponsor wants to know whether it saves time, money or risk. The people who do the work want to know whether it makes their day easier or threatens their job. IT and security want to know where data goes. Find out who is coming before the meeting, and make sure each group hears its question answered in the first ten minutes.

Use the client's words. If they say 'exceptions', do not say 'edge cases'. If they say 'the AP queue', do not say 'the pipeline'.

Handling questions

Questions are good: they show people are imagining using it. Listen to the whole question, repeat it in one sentence to check you understood, then answer briefly. If you do not know, say so and write it down with a date for the answer. Never guess at numbers or promise features on the spot.

When someone asks 'can it also do X?', answer honestly about what exists today, then treat X as a backlog candidate. Ask what problem X would solve; often it reveals a better next step than the one you planned.

Honest limits and live risk

Show at least one case where the solution is wrong or refuses, and explain how the workflow catches it. This builds more trust than a perfect run, because everyone knows AI makes mistakes, and they want to see you know where.

Live demos fail. Rehearse twice, use a stable environment, keep cases ready in a file, and have a recorded run as a backup. If something breaks, switch to the recording calmly and say what happened. Never use real customer data on a shared screen unless the client has agreed.

After the demo, send a short written follow-up the same day: the decision asked for, what was shown, the limits you stated, the open questions with the date you will answer them, and the agreed next step with its owner. People who were not in the room will read that note, and it stops the story from growing in the retelling. If the audience heard 'it is 90 percent accurate' but the eval set had only 30 cases, the note is where you make that clear.

Worked example

Worked example: a 15-minute demo script

Here is a script for the invoice matching assistant. The numbers in it (three clerks, about 400 invoices a week, 60 eval cases) are example values; in a real demo you use the figures you measured in discovery and testing, and you name their source if asked.

Notice how little time goes to the result itself compared with the problem, the evidence and the limits. Notice also that it ends with a yes or no question and a request for an owner, not with 'any questions?'.

text

DEMO SCRIPT - Invoice matching assistant (15 minutes)

1. The problem (2 min)
   "Today three clerks check about 400 invoices a week by hand.
    Here is one from last Tuesday that took 20 minutes to resolve."
2. The result, on their data (6 min)
   - Run the same invoice live. Show the decision and the reason.
   - Show one clean case, one hold, one needs_review.
   - Show the reviewer screen and the audit record.
3. The evidence (3 min)
   - Eval set: 60 real cases from March and April.
   - Pass rate by case type, and the 2 cases it got wrong.
4. The limits (2 min)
   - Not tested on handwritten scans.
   - Every payment still needs a person to approve it.
5. The ask (2 min)
   - Decide: a 4-week pilot with two clerks, yes or no.
   - Name an owner on your side.

Backup: recorded run of steps 2 and 3, saved locally.

Start practicing

Practice 1

Write and rehearse a demo script

60 min

Goal: Produce a 10-minute demo plan that ends in a decision.

Pick something you have built, or the eval exercise from module 9.
Write the one decision you want at the end in a single sentence.
Fill in the five-part script: problem, result, evidence, limits, ask. Keep each part to a few lines.
List the three hardest questions you expect and write a short honest answer to each.
Rehearse out loud with a timer, then once more with a friend or by recording yourself.
Record a backup run of the result part and save it locally.

Done when: You can give the demo in under 10 minutes, it shows at least one limit, and it ends with a clear ask.

I did this

Common mistakes

Opening with architecture diagrams. Start with the client's problem; show the inside only if asked.
Showing only perfect cases. Show one failure and how the workflow handles it.
Using made-up or toy data when real (agreed, masked) data is available.
Ending without an ask, so nothing is decided.
Promising features or dates during questions. Write them down and reply later.

Check yourself

What are the five parts of a decision-focused demo?
Why show a case where the solution fails?
How should you answer a question you do not know the answer to?
What do you prepare in case the live demo breaks?

Lesson 10.2

From POC to pilot to production

You will understand what a proof of concept, a pilot and a production system each prove, who uses each, and what must change between them.

Why this matters on a client engagement

Many AI projects stop after a promising proof of concept because nobody planned the next steps. Others jump straight to production and fail in public. An FDE needs to explain to the client what each stage is for, what it costs, and what has to be true before moving on, so the client can decide with open eyes.

What each stage adds

POC: can it work?Eval set, first build, SME feedback
Pilot: is it worth it?Real users, review step, logs, sign-in
Production: can we rely on it?Client hosting, CI/CD, alerts, support
Owned by the clientRunbook, named owner, trained team

Read top to bottom: each stage keeps everything above it and adds more.

The short version

A proof of concept (POC) answers 'can this work at all on our cases?'. A pilot answers 'does it work for real users in real daily work, and is it worth it?'. Production answers 'can the business rely on it every day, at full volume, safely?'. Each stage proves something different, involves different people, and needs more engineering than the one before.

What each stage proves

Use this as a checklist when you agree scope with the client:

POC: a small build tested on an eval set of real cases. Users: the FDE and one or two SMEs. Proves technical feasibility and gives a first quality number. Weeks, not months.
Pilot: the solution used by a small group in their real work, with a person reviewing outputs. Users: a few named people from the team. Proves value (time saved, errors caught), usability and that the review step works. Has an agreed start, end and success measure.
Production: the solution used by everyone who needs it, as part of the normal process. Users: the whole team, supported by IT. Proves reliability at volume, and that the client can own and run it.

What must change between stages

Moving from POC to pilot usually needs: real sign-in instead of a shared link, data access approved by the client's security team, a review screen and audit log, basic monitoring and cost tracking, and a test environment separate from your dev setup.

Moving from pilot to production usually needs: deployment into the client's own cloud account or platform, CI/CD with an eval gate, alerts with named owners, a runbook, a support model, agreed service levels, backup and recovery, a security and privacy review signed off, and training for every user. It also needs a business owner who is accountable for the outcome after the FDE leaves.

Deciding to move on, or not

Agree the exit criteria for each stage before it starts, in writing. For a pilot that might be a pass rate on the eval set, an edit rate below a limit, a time saving measured against the old process, and positive feedback from the pilot users. At the end, compare results with the criteria and recommend one of three things: move on, run another round with specific changes, or stop.

Stopping is a valid result. A POC that shows the data is too poor, or a pilot that shows the time saved is too small, has saved the client from a larger failed project. Say so plainly, and record what was learned.

Who uses it and who owns it

Ownership should shift as the stages move. In the POC the FDE owns almost everything. In the pilot a business owner on the client side owns the success measures and the user group, while the FDE still runs the system. In production the client owns both the business outcome and the running system, and the FDE has handed over. Plan that shift from the start, or the solution will depend on you forever.

Cost changes across stages too. A POC is cheap to run but cheap in rigour. A pilot adds people's time for review and feedback. Production adds hosting, monitoring, support hours and the cost of keeping the eval set and prompts current. Write a rough estimate for each stage early, using the cost per request you measured, so the sponsor can compare the running cost with the value the pilot shows.

Worked example

Worked example: a stage plan for the invoice assistant

The FDE writes a one-page plan for the client's finance director. The POC runs for a few weeks on 60 real past invoices, with the AP lead labeling cases. Its exit criteria are a pass rate the AP lead agrees is useful and no approve decisions on cases that should have gone to review.

The pilot then runs for four weeks with two clerks using the assistant on live invoices, every payment still approved by a person. Exit criteria: measured time per invoice compared with the old way, reviewer edit rate below an agreed limit, and both clerks willing to keep using it. Before the pilot starts, the client's IT team approves sign-in and the data flow.

Production means all AP staff, hosting in the client's cloud account, alerts owned by the platform team, and the AP lead as business owner. The plan lists each item that must be true before go-live, so nobody is surprised by the effort.

POC: feasibility and a first quality number.
Pilot: value, usability and a working review step.
Production: reliability, ownership and support.

Start practicing

Practice 1

Write a stage plan

45 min

Goal: Turn an idea into a POC, pilot and production plan with exit criteria.

Pick an AI use case you know, such as answering HR policy questions or triaging support tickets.
On one page, write for each stage: what it proves, who uses it, how long it runs.
Write two or three measurable exit criteria for the POC and for the pilot.
List at least six things that must change between pilot and production.
Name who owns the outcome and who owns the system at each stage.
Ask a free LLM chat to challenge your plan as a sceptical IT director, and note any gaps it finds.

Done when: You have a one-page plan with exit criteria for each stage and a named owner at each stage.

I did this

Common mistakes

Calling a POC 'nearly production'. Name the work that remains so the client can budget for it.
Running a pilot with no end date or success measure, so it never ends.
Leaving security and IT review until the week before go-live.
Keeping all ownership with the FDE, so the system dies when you leave.
Treating a decision to stop as a failure instead of a useful result.

Check yourself

What does each of POC, pilot and production prove?
Name five things that must change between pilot and production.
Why agree exit criteria before a stage starts?
How should ownership shift across the stages?

Lesson 10.3

Roadmap, backlog and success measures

You will be able to write a roadmap in terms of outcomes, keep a prioritized backlog using value and effort, and define measures that show whether the work succeeded.

Why this matters on a client engagement

After the first demo, requests pour in. Without a way to decide what comes next, an FDE ends up building whatever the loudest person asked for last. A roadmap tied to outcomes, and a backlog ranked by value and effort, let you say yes and no with reasons the client can see and agree with.

Value versus effort quadrants

High value, low effort

Do first
Often small UX fixes
Builds early trust

High value, high effort

Plan and split up
Needs a design first
Check it still matters

Low value, low effort

Only with spare time
Easy to over-collect
Batch them together

Low value, high effort

Park it
Explain why to requester
Revisit each quarter

Read each column as one quadrant: what goes there and what to do with it.

The short version

A roadmap is a plan of what the team will work on over the coming weeks or months, usually grouped as now, next and later. A backlog is the ordered list of individual items of work. Success measures are the numbers that tell you whether the work achieved what it was for.

The key idea is to plan around outcomes, not outputs. An output is something you build, such as 'a mailbox connector'. An outcome is a change in the business, such as 'clerks no longer download invoices by hand'. Outputs are how you get outcomes.

Outcomes versus outputs

Outcomes keep the work honest. If the outcome is 'invoices are matched within one day of arrival', you may find the best step is not more AI at all but a fix to how invoices reach the mailbox. Writing the outcome first also gives you the success measure for free: you measure the outcome.

A good outcome statement names who benefits, what changes, and how you will know. For example: 'AP clerks spend less time per invoice, measured as average handling time from arrival to decision, compared with the four weeks before the pilot.'

Prioritizing with value versus effort

For each backlog item, estimate value (how much it moves an outcome) and effort (how much work it takes), each on a simple scale such as 1 to 5. Estimate together with the client: they know the value, you know the effort. Then sort:

High value, low effort: do first.
High value, high effort: plan it and break it into smaller pieces.
Low value, low effort: do it only when there is spare time.
Low value, high effort: park it, and tell the requester why.

Writing backlog items

Each item should be small enough to finish in days, not weeks, and should say which outcome it serves. A common format is the user story: 'As an AP reviewer, I want to see the reason for each suggested decision, so that I can approve faster.' Add acceptance criteria: the checks that show it is done, such as 'the reason names the PO line that differs'.

Keep one backlog, visible to the client, in a simple tool they can use: a GitHub project board, a shared spreadsheet, or their own tracker. Review it with the business owner every week or two.

Success measures

Pick a few measures that link to outcomes, and measure the starting point (baseline) before you change anything, or you cannot show improvement. Mix three kinds: business measures (time per case, error rate, backlog of cases), quality measures (eval pass rate, reviewer edit rate), and adoption measures (how many of the intended users use it each week). Leading measures, such as weekly active users, move early; lagging measures, such as cost saved per quarter, prove value later. Report both.

Keep the list short. Three to five measures that the business owner understands and checks are worth more than a dashboard of twenty that nobody opens. Agree who collects each number, how often, and where it is shown, and put that in the hand-over pack so it continues after you leave.

Worked example

Worked example: a small backlog ranked by value and effort

After the pilot demo, the AP team and the FDE list five candidate items, each tied to an outcome, and score them together. The CSV shows the backlog. The Python script ranks the same items by value divided by effort and places each in a quadrant. The scores are judgement calls agreed in the meeting, not measurements, so the ranking is a starting point for discussion, not an answer.

Running the script puts 'Show confidence and reason on each match' first (value 4, effort 1), then the mailbox connector and the weekly report. Single sign-on lands in plan because it has medium value and medium effort but is required to pass IT review, which is a reminder that some items are needed regardless of score. Handwritten scans are parked.

python

# backlog.csv
# id,item,outcome_it_serves,value_1_5,effort_1_5,status
# B1,Read invoices from the shared mailbox,Clerks stop downloading files by hand,5,2,next
# B2,Show confidence and reason on each match,Reviewers decide faster and trust it,4,1,next
# B3,Weekly accuracy report for the finance lead,Owner can see quality without asking,4,2,next
# B4,Single sign-on with the client directory,Pass IT review for the pilot,3,3,planned
# B5,Support scanned handwritten invoices,Cover the last few percent of volume,2,5,parked

backlog = [
    # (item, value 1-5, effort 1-5)
    ("Read invoices from the shared mailbox", 5, 2),
    ("Show confidence and reason on each match", 4, 1),
    ("Single sign-on with the client directory", 3, 3),
    ("Support scanned handwritten invoices", 2, 5),
    ("Weekly accuracy report for the finance lead", 4, 2),
]

def quadrant(value, effort):
    if value >= 4 and effort <= 2:
        return "do first"
    if value >= 3:
        return "plan"
    if effort <= 2:
        return "quick win if time"
    return "park"

for item, value, effort in sorted(backlog, key=lambda b: (-b[1] / b[2], b[0])):
    print(f"{value / effort:4.1f}  {quadrant(value, effort):17}  {item}")

Start practicing

Practice 1

Turn requests into a ranked backlog

45 min

Goal: Practise outcome-first planning and value versus effort ranking.

Write down 8 feature requests for a tool you use at work or for the invoice assistant.
For each, write the outcome it would serve in one sentence. Drop any request with no clear outcome.
In a spreadsheet, score value and effort from 1 to 5 and add value divided by effort.
Sort by that column and label each item with its quadrant.
Write one success measure with a baseline for your top outcome.
Rewrite your top item as a user story with two acceptance criteria.

Done when: You have a sorted backlog where every item names an outcome, and one measure with its baseline.

I did this

Common mistakes

Writing a roadmap as a list of features with dates. Write outcomes and keep dates loose for later items.
Scoring value alone. Value comes from the client; effort from the builders.
Forgetting the baseline, so you cannot prove the change helped.
Treating the value versus effort score as exact. It is a conversation tool.
Keeping the backlog in your own notes where the client cannot see it.

Check yourself

What is the difference between an outcome and an output? Give one of each.
What do you do with a high value, high effort item?
Why must you measure a baseline before the pilot?
What are leading and lagging measures?

Lesson 10.4

The hand-over pack: runbook, owner and support model

You will be able to put together a hand-over pack with a runbook, a named owner and a support model, so the client can run the solution without you.

Why this matters on a client engagement

An FDE engagement ends, but the client's need does not. If the solution only works while you are there, it was a consultancy demo, not a deployment. A clear hand-over pack is what lets the client's team keep it running, fix common problems, and decide its future. It is also the last impression you leave.

Shadow then reverse-shadow hand-over

FDE

Technical owner

Business owner

Writes runbook and RACI draft

Agrees RACI and names owners

Handles issues, owner watches

Handles issues, FDE watches

Runs rollback drill alone

Transfers access and keys

Signs off hand-over

Read top to bottom: support work moves from the FDE to the client team step by step.

The short version

A hand-over pack is the set of documents, access and agreements that move ownership from you to the client. It has three core parts. A runbook: how to operate the system day to day and what to do when things go wrong. A named owner: one person accountable for the business outcome and one team accountable for the running system. A support model: who answers which problem, how fast, and how issues are escalated.

Start building it on day one of the pilot, not in the last week. Every time you fix something, add a line to the runbook.

What goes in a runbook

Write it for a capable person who has never seen the system. Keep it short and practical:

What the system does and does not do, in two or three sentences.
Owners, support hours and contacts.
Where it runs: accounts, repository, deploy workflow, and the names (never values) of its secrets.
Routine checks: which dashboard to look at, and the weekly eval rerun.
Common problems: symptom, first check, fix or escalate.
Release and rollback steps.
Known limits and open risks.

A RACI for support

RACI is a simple table that shows, for each task, who is Responsible (does the work), Accountable (owns the result and signs off; only one per task), Consulted (asked for input before) and Informed (told after). It removes the most common hand-over problem: everyone assuming someone else will handle it.

For an AI solution, typical rows are: answer user questions, investigate a wrong answer, update the prompt or eval set, approve a release, renew API keys, watch cost, and decide on new features. Fill it in with the client in a meeting, not alone, and make sure each Accountable name agrees.

text

Task                        AP lead  Platform team  IT security  FDE (until exit)
Answer user questions       A        R              -            C
Investigate a wrong answer  A        R              -            C
Update prompt or eval set   A        R              -            C
Approve a release           A        R              C            I
Rotate API keys             I        A/R            C            -
Watch cost and alerts       I        A/R            -            -
Decide on new features      A        C              C            I

The support model

Agree tiers. First line is usually the business team or the client's service desk: they handle how-to questions and known issues from the runbook. Second line is the technical owner: they investigate errors, wrong answers and alerts. Third line, if any, is the vendor or the original builders, for problems nobody else can fix. Write down how users report a problem, how fast each tier responds, and what counts as urgent.

Then run a shadow period: for a week or two the client's team handles every issue while you watch and only step in when asked. Then reverse it so you are not involved at all, and see what breaks. Every gap you find goes into the runbook.

Closing the engagement

Before you leave, check access: the client must own every account, repository and key, and your personal access should be removed. Hand over the eval set and results history, the backlog with its priorities, and a short note on open risks. Hold a final session where the owner walks you through a routine check and a rollback. If they can do it without help, the hand-over has worked.

Agree how the client can reach you or your team after the engagement, if at all, and for how long. A clear end date with a short, named period of light support is better than an open promise. It tells the owner they are now in charge, and it gives them a safety net while they get used to it.

Worked example

Worked example: a runbook template

Here is a short runbook for the invoice matching assistant, written in Markdown so it can live in the same repository as the code. Placeholders such as 'Repo, branch, and deploy workflow name' are filled with the real values for the client. It names secrets but never contains their values.

The common problems table is the part support teams use most. Each row starts from what a person notices (the symptom), not from the technical cause, because that is how problems are reported.

markdown

# Runbook: Invoice matching assistant

## What it does
Suggests approve, hold, reject or needs_review for each invoice. A person approves every payment.

## Owners
Business owner: AP lead. Technical owner: platform team. Support hours: Mon-Fri 8-18.

## Where it runs
Container service in the client cloud account. Repo, branch, and deploy workflow name.
Secrets: names only (ANTHROPIC_API_KEY in the secret store). Never the values.

## Daily and weekly checks
- Dashboard: error rate, latency, daily cost, reviewer edit rate.
- Weekly: rerun eval set; add new failure cases.

## Common problems
| Symptom | First check | Fix or escalate |
| --- | --- | --- |
| Queue not moving | Mailbox connector log | Restart job; escalate to IT if mailbox locked |
| Cost alert | Requests per hour, retries | Pause batch job; tell technical owner |
| Edit rate rising | Last 20 edited cases | Add to eval set; review prompt with AP lead |

## Release and rollback
Merge to main, deploy to test, AP lead checks 10 cases, promote to prod.
Rollback: redeploy the previous tagged build.

## Contacts
Named people and how to reach them, kept current by the business owner.

Start practicing

Practice 1

Write a runbook and RACI

60 min

Goal: Produce a hand-over pack someone else could use without you.

Pick a small tool you built or use, even a spreadsheet with formulas or a script.
Copy the runbook template above into a Markdown file and fill in every section.
Write at least three common problems with symptom, first check and fix.
Draw a RACI table with five tasks and three or four roles; give each task exactly one A.
Give the runbook to a friend and ask them to do one routine check using only the document.
Fix every place where they got stuck.

Done when: Another person completed a routine check from your runbook alone, and every task in your RACI has exactly one Accountable.

I did this

Common mistakes

Writing the runbook in the last week. Build it as you go.
Naming a team as Accountable instead of a person.
Putting secret values in the runbook. Name the secret and where it is stored.
Leaving accounts or keys in your own name after you leave.
Skipping the shadow period, so the first real incident is the first test.

Check yourself

What are the three core parts of a hand-over pack?
What do R, A, C and I stand for, and why only one A per task?
What belongs in the common problems section of a runbook?
How do you know the hand-over has worked?