Building blocks: code, APIs, data and Git

The short version

The web is a conversation between two programs. The client asks for something and the server answers. Your browser is a client. A Python script that calls an API is also a client. The server is a program on another computer that waits for requests and sends back responses.

Think of it like an internal request form in a company. You fill in a form with what you want (the request), send it to the right department (the server), and get back a reply with a result code and some content (the response). HTTP, the Hypertext Transfer Protocol, is the agreed format for that form. Because everyone uses the same format, any client can talk to any server.

Step by step: from address to page

Take the address https://shop.example.com/orders. It has three parts: the scheme (https), the host name (shop.example.com) and the path (/orders). Here is what happens when you press Enter.

• DNS lookup. Computers find each other by IP address, a number such as 93.184.215.14. The Domain Name System (DNS) is the internet's phone book. Your computer asks a DNS resolver for the IP address of shop.example.com and gets one back.
• Connection. The browser opens a connection to that IP address. With https it also runs TLS (Transport Layer Security): the server shows a certificate that proves who it is, and both sides agree on encryption keys.
• Request. The browser sends an HTTP request: a method (GET), the path (/orders), and headers such as which browser it is and which formats it accepts.
• Response. The server sends back a status code (200 means OK), response headers (for example Content-Type: text/html) and a body, the actual content.
• More requests. The HTML page mentions images, stylesheets and scripts. The browser sends a separate request for each one, which is why one page can make dozens of requests.

Methods, status codes and headers

The method says what you want to do. GET reads data. POST creates something new or submits data. PUT replaces a record and PATCH changes part of one. DELETE removes it. A safe habit: GET should never change anything on the server.

The status code is a three digit number that tells you how it went. Learn the families, not every number. 2xx means success (200 OK, 201 Created). 3xx means look elsewhere (301 moved permanently). 4xx means the client made a mistake: 400 bad request, 401 not authenticated, 403 not allowed, 404 not found, 429 too many requests. 5xx means the server failed: 500 internal error, 503 unavailable. The family tells you where to look first: 4xx means check what you sent, 5xx means the other side has a problem.

Headers are name and value pairs that carry extra information. Request headers include Authorization (who you are), Content-Type (the format of the body you are sending) and Accept (the format you want back). Response headers include Content-Type, Cache-Control (how long the response can be reused) and Set-Cookie.

HTTPS and why it matters

Plain HTTP sends everything as readable text, so anyone on the network path could read passwords or customer data. HTTPS is HTTP sent inside a TLS encrypted connection. The certificate check also proves you are talking to the real server and not an impostor. For client work the rule is simple: anything with credentials or business data goes over HTTPS only. If a client system only offers plain HTTP, raise it as a risk.

Where this shows up on an engagement

Inside a client's company the path between client and server is often longer than on the public internet. Requests may pass through a VPN, a corporate proxy or a firewall before they reach the server. A call that works from your laptop at home can fail on the client's network with a timeout, a certificate error or a 407 status (proxy authentication required). When that happens, the code is usually fine; the network path is different. Ask the client's IT team early which outbound addresses are allowed, especially for cloud LLM APIs.

Two more ideas come up all the time. A cookie is a small piece of data the server asks the browser to store and send back on later requests; it is how a site remembers that you are logged in. A session is the server's record of that logged in state. When you automate a system that normally needs a browser login, you usually need an API key or token instead, because scripts do not log in the way people do.

Finally, keep the browser's developer tools in mind as a free debugging tool. The Network tab shows every request a web application makes, which often reveals the internal API behind a screen the client uses every day. That can be the fastest route to the data you need for a proof of concept, but always ask the system owner before you call it directly.

Going deeper: see it from the command line

curl is a command line client installed on macOS, Linux and recent Windows. It lets you send a request and see the raw response without a browser. The -I option asks for headers only. nslookup shows the DNS step on its own.

nslookup example.com
curl -I https://example.com
curl -i https://example.com

The short version

A Python script is a list of instructions the computer runs from top to bottom. If you have written a spreadsheet formula, you already think this way: take some values, apply a rule, produce a result. Python lets you do the same thing on thousands of rows, repeat it every day, and connect it to other systems.

You run a script by saving it as a file, for example report.py, and typing python3 report.py in a terminal. Lines starting with # are comments for humans; Python ignores them.

Variables and types

A variable is a named box that holds a value: supplier = "Acme Parts". The type says what kind of value it is. The four you will use every day are str (text, in quotes), int (whole numbers), float (decimal numbers) and bool (True or False).

Types matter because Python treats them differently. "120" + "5" gives the text "1205", while 120 + 5 gives 125. Data read from a CSV file always arrives as text, so you must convert numbers with float() or int() before doing maths. This is the most common beginner bug in data scripts.

Lists, dicts, functions and loops

A list holds items in order, like a single column: amounts = [120.0, 75.5, 300.0]. A dict maps keys to values, like one row with named columns: {"id": "INV-001", "amount": 120.0}. You read a value with its key: invoice["amount"]. A list of dicts is the Python shape of a table, and it is exactly what most APIs return.

A function is a named, reusable rule. You define it once with def, give it inputs, and it returns an output. A for loop repeats the same steps for each item in a list. An if statement runs steps only when a condition is true. Indentation (four spaces) shows which lines belong inside the function, loop or if.

# Variables and types
supplier = "Acme Parts"      # str (text)
invoice_count = 3            # int (whole number)
vat_rate = 0.2               # float (decimal number)
is_approved = False          # bool (True or False)

# A list keeps items in order
amounts = [120.0, 75.5, 300.0]

# A dict maps keys to values, like one row of a spreadsheet
invoice = {"id": "INV-001", "supplier": supplier, "amount": 120.0}

def add_vat(amount, rate=vat_rate):
    """Return the amount including VAT."""
    return round(amount * (1 + rate), 2)

total = 0
for amount in amounts:
    with_vat = add_vat(amount)
    print(f"{amount} becomes {with_vat}")
    total += with_vat

print("Total with VAT:", total)
print(invoice["supplier"], "has", invoice_count, "invoices")
if total > 500 and not is_approved:
    print("Needs approval")

Reading CSV and JSON files

CSV (comma separated values) is the format almost every business system can export. JSON (JavaScript Object Notation) is the format of APIs and config files: text made of objects in curly braces and lists in square brackets. Python reads both with built in modules, so nothing extra needs installing.

Use with open(...) as f: to open a file; it closes the file for you when the block ends. csv.DictReader turns each CSV row into a dict keyed by the header line. json.load turns JSON text into dicts and lists, and json.dump writes them back out. The two small files below are used in the worked example; save them exactly as shown.

invoices.csv
invoice_id,supplier,amount,status
INV-001,Acme Parts,120.00,paid
INV-002,Brightline Ltd,75.50,open
INV-003,Acme Parts,300.00,open

config.json
{"approval_limit": 250, "currency": "GBP", "approvers": ["finance", "ops"]}

Reading error messages

Every script you write will fail at some point, and that is normal. When Python stops with an error it prints a traceback: the chain of lines that led to the problem. Read it from the bottom up. The last line names the error type and gives a short message; the lines above show the file and line number where it happened.

A few error types cover most of what you will see in data work. FileNotFoundError means the file is not where the script looked, often because you ran the script from a different folder. KeyError: 'amount' means a dict has no key with that name, usually a column header spelled differently, with a space, or with different capital letters. ValueError: could not convert string to float means a value such as 'n/a' or an empty cell reached float(). TypeError usually means you mixed text and numbers, for example adding a str to a float. NameError means you used a variable before creating it or misspelled it.

When you are stuck, add print() lines to show the values just before the failing line, then run again. Change one thing at a time. You can also paste the full traceback, with no confidential data in it, into an AI assistant and ask what it means, but check its fix by running the script yourself.

Virtual environments and pip

Python's built in modules cover a lot, but you will often need extra packages, such as requests for calling APIs. pip is the tool that installs them. A virtual environment is a private folder of packages for one project, so two projects can use different versions without breaking each other. Create one per project, activate it before you work, and record the packages in requirements.txt so anyone can rebuild the same setup with pip install -r requirements.txt.

python3 -m venv .venv              # create the environment once per project
source .venv/bin/activate          # Windows: .venv\Scripts\activate
python -m pip install requests     # install a package into this project only
python -m pip freeze > requirements.txt
python -c "import requests; print(requests.__version__)"
deactivate                         # leave the environment

The short version

An API (Application Programming Interface) is a way for one program to ask another program for data or actions. A web API uses HTTP, the same request and response you saw in the previous lesson. Instead of an HTML page, the server returns data, usually as JSON.

REST is the most common style. Each kind of thing has an address, called an endpoint, for example /customers or /orders/4471. You use HTTP methods on those endpoints: GET /orders lists orders, GET /orders/4471 reads one, POST /orders creates one, PATCH /orders/4471 changes one. The API's documentation lists every endpoint, the parameters it accepts and the shape of the JSON it returns. Reading that documentation first saves hours.

Endpoints, query parameters and JSON

Query parameters filter or shape a request. They go after a ? in the address, joined by &: /todos?userId=1&completed=false. In Python you pass them as a dict and the library builds the address for you, including escaping spaces and special characters.

JSON is text with a small set of building blocks: objects in curly braces with "key": value pairs, lists in square brackets, strings in double quotes, numbers, true, false and null. In Python, objects become dicts and lists become lists, so response.json()[0]["title"] means the title of the first item.

Authentication and API keys

Most real APIs need to know who is calling. The usual way is an API key or token sent in a header, often Authorization: Bearer YOUR_API_KEY or a vendor header such as x-api-key. A key is like a password: anyone who has it can act as you. Keep keys in environment variables or a .env file that is never committed, never paste them into chat tools or tickets, and ask the client for a key with the least access the proof of concept needs.

A 401 status usually means the key is missing or wrong. A 403 means the key is valid but not allowed to do this action.

Errors, rate limits and pagination

Always check the status code before using the body. A failed call often still returns JSON, just an error message instead of data. In Python, response.raise_for_status() stops the script with a clear error on any 4xx or 5xx.

Rate limits cap how many calls you can make in a period. When you go over, the API returns 429 Too Many Requests, often with a Retry-After header saying how many seconds to wait. Many APIs also send headers showing how many calls you have left. Respect them: a proof of concept that hammers a client's system will get its key blocked.

Pagination means a big list is split into pages. You ask for page 1, then page 2, until a page comes back empty or the API says there are no more. If you forget, you silently process only the first page and your totals are wrong. Different APIs page differently (page numbers, offsets, cursors or a next link), so check the documentation.

import time
import requests

BASE_URL = "https://jsonplaceholder.typicode.com"

def get_with_retry(url, params, attempts=3):
    """GET a URL, waiting and retrying if the API says slow down (429)."""
    for attempt in range(attempts):
        response = requests.get(url, params=params, timeout=10)
        if response.status_code == 429:
            wait = int(response.headers.get("Retry-After", "5"))
            print(f"Rate limited, waiting {wait} seconds")
            time.sleep(wait)
            continue
        response.raise_for_status()  # stops with an error on 4xx or 5xx
        return response
    raise RuntimeError("Gave up after repeated rate limits")

all_posts = []
page = 1
while True:
    resp = get_with_retry(f"{BASE_URL}/posts", {"_page": page, "_limit": 25})
    batch = resp.json()
    if not batch:  # an empty page means we have everything
        break
    all_posts.extend(batch)
    print(f"Page {page}: {len(batch)} posts")
    page += 1

print("Total posts:", len(all_posts), "| API says:", resp.headers.get("X-Total-Count"))

Habits for proof of concept integrations

Before you write a loop that calls an API thousands of times, make one call by hand with curl and read the whole response. Then make the same call from Python and print it. Only then build the loop, and run it first with a small limit such as two pages.

Keep the key out of the code. Set it as an environment variable in your terminal (export MY_API_KEY=YOUR_API_KEY on macOS or Linux) and read it in Python with os.environ["MY_API_KEY"]. This means the script can be shared and committed without the secret, and the client's team can use their own key after handover.

Save raw responses to a file while you develop. You can then work on the processing steps without calling the API again, which is faster, avoids rate limits and gives you test data. Log every call's status code and timing so you can show the client what happened when something fails.

Be careful with calls that change data. A GET can be repeated safely, but repeating a POST may create the same order or ticket twice. Before writing to a client system, agree with its owner which test environment to use.

Trying it with curl

JSONPlaceholder is a free fake REST API for practice; it needs no key. Its POST requests return a realistic reply but nothing is actually saved. The first command below lists two tasks; -i shows the status line and headers. The second sends JSON with POST.

curl -i "https://jsonplaceholder.typicode.com/todos?userId=1&_limit=2"

curl -X POST https://jsonplaceholder.typicode.com/posts \
  -H "Content-Type: application/json" \
  -d '{"title": "Late delivery", "body": "Order 4471 arrived 3 days late", "userId": 1}'

The short version

A relational database stores data in tables. A table is like a spreadsheet tab with fixed columns: each row is one record (one invoice), and each column is one field (amount, status). SQL (Structured Query Language) is the language for asking questions of those tables. The same basic SQL works in SQLite, PostgreSQL, SQL Server, Snowflake and most other databases, with small differences.

SQLite is a full database stored in a single file, with nothing to install on macOS and most Linux systems. It is ideal for learning and for small proofs of concept. Python can also use it through its built in sqlite3 module.

You do not need to become a database administrator. For an FDE the goal is narrower: read a client's tables, answer questions from them, check that the numbers match what the business sees in its reports, and explain any difference. Those skills transfer to whatever database the client runs.

Tables, rows and keys

A primary key is a column whose value is unique for every row, such as invoice_id. It is how you point at exactly one record. A foreign key is a column in one table that holds the primary key of another table. In the example below, invoices.supplier_id points to suppliers.supplier_id. This lets you store each supplier's details once instead of repeating them on every invoice.

When you meet a new client database, your first job is to find the tables that matter, their primary keys, and how they link. Ask for a data dictionary or an entity relationship diagram; if none exists, that is worth knowing too.

Asking questions: SELECT, WHERE, JOIN, GROUP BY

SQL describes what you want, not how to get it. The database works out the steps. A useful habit is to write the question in plain words first, for example 'open invoice total per supplier, biggest first', then translate each phrase into a clause.

• SELECT picks the columns you want. FROM names the table.
• WHERE keeps only the rows that match a condition, like a filter in a spreadsheet. Text goes in single quotes.
• ORDER BY sorts the result. DESC means largest first.
• JOIN combines rows from two tables where the keys match, so each invoice can show its supplier's name.
• GROUP BY collapses rows into one row per group, with totals from COUNT, SUM, AVG, MIN or MAX. It is a pivot table in SQL.
• HAVING filters groups after they are totalled, while WHERE filters rows before.

Building a small database

CREATE TABLE defines a table and its columns. INSERT adds rows. Save the script below as setup.sql and run sqlite3 shop.db < setup.sql. This creates the file shop.db with two linked tables. The numbers are made up for practice.

CREATE TABLE suppliers (
  supplier_id INTEGER PRIMARY KEY,
  name        TEXT NOT NULL,
  country     TEXT
);

CREATE TABLE invoices (
  invoice_id  TEXT PRIMARY KEY,
  supplier_id INTEGER NOT NULL REFERENCES suppliers(supplier_id),
  amount      REAL NOT NULL,
  status      TEXT NOT NULL,
  issued_on   TEXT NOT NULL
);

INSERT INTO suppliers (supplier_id, name, country) VALUES
  (1, 'Acme Parts', 'UK'),
  (2, 'Brightline Ltd', 'IE'),
  (3, 'Northwind Foods', 'UK');

INSERT INTO invoices (invoice_id, supplier_id, amount, status, issued_on) VALUES
  ('INV-001', 1, 120.00, 'paid', '2025-01-05'),
  ('INV-002', 2,  75.50, 'open', '2025-01-09'),
  ('INV-003', 1, 300.00, 'open', '2025-02-02'),
  ('INV-004', 3, 980.00, 'open', '2025-02-14'),
  ('INV-005', 2, 410.00, 'paid', '2025-02-20');

Data problems to look for first

Real client data is messier than practice data, and SQL is the quickest way to find out how messy. Run a few checks before you build anything on top of a table.

Count the rows, then count the distinct values of the key: SELECT COUNT(*), COUNT(DISTINCT invoice_id) FROM invoices. If the numbers differ, there are duplicates, and every total built on that table will be wrong. Look for empty values: in SQL a missing value is NULL, and you test for it with IS NULL, not = NULL. Check the smallest and largest values of amounts and dates with MIN and MAX; a negative amount or a date in the year 1900 tells you something about how the source system is used.

An ordinary JOIN keeps only rows that match on both sides, so an invoice with an unknown supplier silently disappears from the result. A LEFT JOIN keeps every row from the first table and fills the missing side with NULL, which makes these gaps visible. Dates are another trap: SQLite stores them as text, so they only sort and compare correctly in the year-month-day format used in this lesson. Write down every problem you find. A short list of data quality issues is valuable to the client on its own, and it explains later why some numbers differ from their reports.

Using SQL from Python

In a proof of concept you often run queries from a script. Use ? placeholders for values instead of building the SQL text yourself. This prevents SQL injection, where input text is mistaken for SQL commands, and handles quotes in values for you. Run this after setup.sql; it prints INV-003 300.0 and INV-004 980.0.

import sqlite3

conn = sqlite3.connect("shop.db")
rows = conn.execute(
    "SELECT invoice_id, amount FROM invoices WHERE status = ? AND amount > ?",
    ("open", 250),  # placeholders keep user input out of the SQL text
).fetchall()
for invoice_id, amount in rows:
    print(invoice_id, amount)
conn.close()

The short version

Git is version control: it records snapshots of a folder of files over time. Think of the 'track changes' feature in a document, but for a whole project, with a clear label on every change. GitHub is a website that hosts Git repositories so a team can share them, discuss changes and review them before they go in.

Git runs on your computer and works without the internet. GitHub is a separate service you push your history to. You need both words, because people mix them up.

The core ideas

Most teams protect the main branch so nobody can push to it directly. Every change arrives through a reviewed pull request.

• Repository (repo): a project folder whose history Git tracks. It lives on your machine and usually also on GitHub.
• Commit: a saved snapshot with a message that says what changed and why. Small commits with clear messages make history easy to read and easy to undo.
• Staging: before a commit you choose which changed files go in with git add. This lets you leave out files that are not ready.
• Branch: a separate line of work. You create a branch for each change so the main branch stays working while you experiment.
• Push and pull: push sends your commits to GitHub; pull brings other people's commits to your machine.
• Pull request (PR): a request on GitHub to merge your branch into main. It shows the exact lines changed and is where review happens.
• Merge: bringing a branch's commits into another branch, normally after the pull request is approved.

Reviews and pull requests

A pull request is a conversation about a change. Write a title that says what it does, and a description that says why, how you tested it and anything the reviewer should look at closely. Keep it small: a reviewer can check 50 lines properly but will skim 2,000. Reviewers leave comments on specific lines, you push more commits to the same branch to address them, and once approved the branch is merged. Automated checks, such as tests, often run on every pull request too.

When you review someone else's pull request, read the description first, then the changes, and ask questions rather than making demands. The goal is a correct change, not a perfect one.

Commands you will use every day

You can do most daily work with about ten commands. Each one is typed in a terminal inside the project folder. Editors such as VS Code and Cursor also have buttons for all of them, but knowing the commands helps you understand what the buttons do.

• git clone URL: copy a repository from GitHub to your machine, with its full history.
• git pull: bring in new commits from GitHub for the branch you are on.
• git status: show which files are changed, staged or untracked. Run it all the time.
• git switch -c NAME: create a new branch and move to it. git switch NAME moves to an existing branch.
• git add FILE and git commit -m "message": stage a file, then save a snapshot.
• git diff: show the lines you changed that are not yet staged.
• git push -u origin NAME: send your branch to GitHub under its own name.
• git log --oneline: show the history, one line per commit.
• git restore FILE: throw away uncommitted changes to a file. Use it with care; those changes are gone.

Keeping secrets out

Anything committed stays in the history, even if you delete the file in a later commit. If an API key, password or customer data file is pushed, treat it as leaked: tell the owner, revoke the key and create a new one. Deleting it from the repo is not enough.

Prevent this with a .gitignore file, a list of file patterns Git should never track. Put secrets in a .env file and list .env in .gitignore. Also ignore virtual environments, local databases and large data exports. Run git status before every commit and read the list of files. GitHub also offers secret scanning, which can warn when a known key format is pushed, but it is a safety net, not a plan.

At handover, the repository is the product. Make sure the README says how to install and run the work, which settings and keys are needed (by name, never by value) and where the data comes from. A client engineer should be able to clone the repository and run it without calling you.

The short version

An AI coding assistant is an LLM connected to your code. It can read files, suggest changes, write new code, run commands and explain what code does. Three common tools work in slightly different ways. GitHub Copilot sits inside your editor and suggests code as you type, with a chat panel for bigger requests. Cursor is a code editor built around AI, with an agent that can edit several files. Claude Code is an agent that works in your terminal or IDE: it reads the project, plans, edits files and runs commands such as tests, asking permission as it goes.

They all follow the same pattern: you describe what you want, the assistant proposes changes, and you accept, reject or ask for changes. Treat it like a fast, well read junior colleague who has never seen this client before. It needs context, clear instructions and review.

Plan, write, test, explain

Give it context the way you would brief a contractor: what the business process is, an example of real input, what good output looks like, and any rule it must follow. Claude Code and Cursor can also read a project instruction file (CLAUDE.md for Claude Code, rules files for Cursor) so you do not repeat the same context every time.

• Plan: before any code, ask the assistant to read the relevant files and propose a plan. Correct the plan; it is much cheaper than correcting code.
• Write: ask for one small change at a time, with the inputs, outputs and rules spelled out.
• Test: ask for tests that cover normal cases, edge cases and bad input, and run them yourself. Tests are how you know it works, not the assistant's word.
• Explain: ask it to explain any line you do not understand. If you cannot explain the code to the client, you are not ready to hand it over.

How to review AI-written code

Read every change as a diff before accepting it. Use this checklist each time:

• Does it do what I asked, and only that? Look for extra changes you did not request.
• Do I understand every line? Ask for an explanation of anything unclear.
• Did I run it, and did the tests pass on my machine? Do the tests check real cases or just restate the code?
• What happens with blank, wrong or unexpected input? Does it fail loudly or guess silently?
• Does it invent anything? Check that every library, function, API endpoint and field name really exists in the docs.
• Are secrets, customer data and keys kept out of code, logs and prompts?
• Would the client's engineers be comfortable maintaining this?

When the assistant gets stuck

Sometimes the assistant goes in circles: each fix breaks something else, or it keeps saying the problem is solved when it is not. Stop and change the approach instead of repeating the same request. Paste the exact error message and the command you ran. Ask it to explain what it thinks is wrong before it changes anything. Break the task into a smaller step you can test on its own.

Long conversations also drift. The assistant can lose track of earlier decisions or mix in old versions of a file. Starting a fresh session with a short summary of the goal, the current state and the next step often works better than pushing on. Claude Code's best practices guide recommends this: run /clear between unrelated tasks, and after two failed corrections, clear the session and write a better first request using what you learned.

Finally, know when to stop using it for a task. If you cannot judge whether the answer is right, because the topic is new to you, use the assistant to explain the concept and point you to official documentation, then decide yourself. The assistant speeds up work you can check. It does not replace your judgement about what the client needs.

Working safely with client code

Check the client's policy before you use any AI tool on their code or data. Some clients allow only approved tools or accounts, and some forbid sending certain data outside their network. Never paste customer records, keys or confidential documents into a prompt to 'just test'. Use made up sample data instead, shaped like the real data, so the code still meets the real cases. Keep Git as your safety net: commit before asking the assistant for a big change, so you can see exactly what it did and undo it.