Performance

Who This Is For

Anyone who builds things with code and wants to understand why they're sometimes slow, expensive, or burning through resources faster than expected. This includes AI-specific performance — why some AI integrations cost far more than they should and how to think about token usage as a real engineering constraint.

No deep technical background required. This guide is about mental models and patterns, not micro-optimization. It's a 35-minute read.

Why Performance Thinking Matters Now

Performance used to feel like an advanced topic — something you worried about after you'd built the thing and proven it worked. AI changes the calculus.

Every call to an AI API costs money and takes time, and both scale directly with how you use it. A poorly designed AI workflow can cost ten times more than a well-designed one that produces the same results. Latency that's tolerable in a batch job is unacceptable in a user-facing product.

Performance thinking in the AI era isn't just about making code fast. It's about understanding the economics of what you're building.

The Four Root Causes

1. Doing Too Much Work

The program is doing more computation than necessary. A loop that iterates a million times when a hundred would do. A query that scans an entire table when an index could jump to the right rows. An algorithm that compares every item to every other item when a smarter approach exists.

2. Doing Work Repeatedly

Computing the same thing multiple times. The same database query runs on every request, fetching data that hasn't changed. The same expensive calculation runs every time a value is needed, even though the inputs haven't changed.

3. Waiting for Things

The program is fast, but it spends most of its time waiting. Waiting for a network response. Waiting for a database query. Waiting for an AI API to return. A function that takes a microsecond to execute but waits 200ms for a database response is a 200-millisecond function.

4. Moving Too Much Data

Retrieving or transmitting more data than needed. A query that fetches every column when two are needed. An AI prompt that includes entire documents when a summary would do. Moving data takes time and money.

The Shape of Slowness

Big O notation describes how performance scales as the amount of work grows.

Quadratic Time — The One to Spot

The classic case: a loop inside a loop, where the inner loop iterates over all items for each item in the outer.

# O(n²): for every user, check every other user
for user_a in users:
    for user_b in users:
        if user_a.interests == user_b.interests:
            suggest_connection(user_a, user_b)

With 100 users: 10,000 operations. With 10,000 users: 100,000,000 operations. This is the shape of code that works fine in development (small data) and collapses in production (real data).

The Practical Takeaway

You don't need to calculate Big O for every function. Recognize the pattern: does this nested loop grow quadratically? Is this operation scanning everything, or jumping to the right place? When data scales from thousands to millions, what happens?

Latency Is the Tax on Every Network Call

Every time your code makes a network request, there's a minimum cost just for the round trip. A request in the same data center: 1–2ms. Across the internet: 50–200ms. To an AI API: 500ms to several seconds.

Round Trips Add Up

If a page load requires ten sequential network requests, you're paying latency ten times. 10 × 100ms = 1 second of unavoidable waiting, before a single byte of data processing.

The pattern to watch: sequential dependencies. Request A, wait, then B, wait, then C. If B and C don't depend on each other, run them in parallel — total wait becomes the longest single wait, not the sum.

Caching — Pay the Cost Once

If a network response doesn't change between requests, cache it: store the result and return it directly on subsequent requests. Caching is the most impactful performance technique in most web applications.

The tricky part is knowing when cached data is stale. This is why "cache invalidation" is famously one of the hardest problems in computing.

Database Performance

Database problems are extremely common because databases sit at the intersection of all four root causes: too much work, repeated work, waiting, and moving too much data.

The N+1 Problem

Fetch 100 orders:                      1 query
For each order, fetch the user:        100 queries
Total:                                 101 queries

At 100 orders: ~200ms. At 10,000 orders: ~20 seconds. At 1,000,000: effectively broken.

The fix: fetch related data in bulk with a JOIN or WHERE IN. AI-generated code is particularly prone to N+1 patterns. For a thorough treatment of JOINs, query structure, indexing, and SQL pitfalls, see the SQL field guide.

Missing Indexes

A missing index turns an O(log n) lookup into an O(n) scan. On a million-row table: microseconds vs. seconds. Most commonly missing on:

• Foreign key columns (user_id, order_id)
• Columns frequently used in WHERE clauses
• Columns used in ORDER BY on large result sets

Fetching More Than You Need

SELECT * fetches every column. If a table has 30 columns and you need 3, you're fetching 10x the data. Same for rows: fetching 10,000 to display 20 is wasteful. Paginate at the database level.

Queries Inside Loops

Any database query inside a loop is a red flag. 1,000 queries in a loop = at minimum a couple of seconds. Almost always fixable with a single query that fetches everything needed upfront.

Doing Work Twice

Memoization

Store the result of a function call and return the cached result on subsequent calls with the same inputs.

from functools import lru_cache

@lru_cache(maxsize=256)
def get_user_permissions(user_id: int) -> list[str]:
    # This query only runs once per unique user_id
    return db.query(
        "SELECT permission FROM user_permissions WHERE user_id = ?",
        user_id
    )

Without the cache: query runs every call. With it: runs once per unique user_id. If called ten times per request for the same user, the query runs once instead of ten.

What's Worth Caching?

The trade-off: memory. Caches need size limits and expiration policies, or they grow until they consume all available memory.

Doing Work You Don't Need To

Lazy Evaluation — Compute When Needed

# Eager: fetches all 10,000 users into memory before filtering
all_users = db.query("SELECT * FROM users")
active_users = [u for u in all_users if u.active]

# Lazy: only fetches rows matching the condition
active_users = db.query("SELECT * FROM users WHERE active = true")

Short-Circuiting

Many operations can stop early. Most languages short-circuit and / && — they stop evaluating as soon as they find a false condition.

# Put cheap checks before expensive ones
def is_valid(data):
    return (
        check_format(data) and          # fast
        check_required_fields(data) and  # fast
        check_business_rules(data)       # expensive (DB call)
    )

Pagination

If you're displaying 20 results per page, don't load 10,000 records. This applies at every layer: database queries should have LIMIT clauses, APIs should return paginated results, UIs should request only what they're showing.

Token Economics

Language models process text as tokens — roughly, word fragments. You pay for tokens twice: the input (your prompt) and the output (the model's response). Both cost money and take time. Prompt design directly affects token usage — for techniques that produce better results with fewer tokens, see the Prompting field guide.

The Most Expensive Pattern: Stuffing Context

The single biggest source of unnecessary AI costs is sending more context than the model needs. Every token in the prompt costs money and takes time to process.

Common versions of this mistake:

• Sending entire conversation history when only the last few exchanges are relevant
• Sending entire documents when the model only needs specific sections
• Sending raw data when a summary would do — 500 rows of sales data vs. aggregated figures
• Boilerplate system prompts that are 80% irrelevant to most requests

Context Window Design

Design for context efficiency by asking at each step:

• What information does the model genuinely need for this task?
• What can be computed before the AI call and passed as a compact summary?
• What can be retrieved on-demand (via tool calls) rather than loaded upfront?

AI Latency Patterns

AI API calls are slow — typically 500ms to several seconds for a standard response. Patterns that compound this:

Sequential AI calls — calling AI for step 1, waiting, then step 2. If steps are independent, run them in parallel. If truly sequential, consider combining into one call.

Retry loops without backoff — immediate retries add to congestion. Use exponential backoff (1s, 2s, 4s).

Generating then discarding — generating a long response and using only part of it. Constrain output length in your prompt and parameters.

Using AI for non-AI tasks — extraction, reformatting, and simple pattern matching that a regular function could handle faster and cheaper.

AI Optimization Techniques

Output Length Control

Output tokens cost money and increase latency. Without constraints, models default to thorough (verbose) responses.

"Respond in one sentence."
"Give me only the JSON object, no explanation."
"List the three most important points, no more."

Caching AI Responses

For AI calls that produce the same result for the same input — same classification, same document summary, same transformation — cache at the application layer.

Cache: document summaries, entity extraction from fixed data, classification of recurring inputs. Don't cache: personalized responses, time-sensitive content, creative tasks where variation is valuable.

Model Selection as a Performance Lever

What Users Feel

The Critical Rendering Path

When a browser loads a page: download HTML, parse it, download CSS and JavaScript, execute JavaScript, calculate layout, paint pixels. Anything that blocks this path delays when the user sees something.

The key metrics:

• First Contentful Paint — when the user sees anything
• Largest Contentful Paint — when the main content is visible
• Time to Interactive — when the page responds to input

Render-Blocking Resources

JavaScript and CSS in the <head> block the browser from showing anything until fully downloaded. For non-critical scripts, use async or defer attributes. This is one of the easiest and highest-impact frontend optimizations.

Image Performance

Images are often the largest assets on a page. Key practices: appropriate sizing (don't send a 4MB photo to display at 200px), modern formats (WebP), and lazy loading (don't download images until they're about to scroll into view).

Perceived Performance

Users tolerate slow operations when they can see progress:

• Skeleton screens feel faster than spinners
• Streaming AI responses feel faster than waiting for the complete response
• Optimistic updates (show the result immediately, revert if it fails) feel faster than waiting for server confirmation

Don't Guess — Measure

Timing

The simplest profiling: record time before and after an operation.

import time

start = time.time()
result = expensive_operation()
elapsed = time.time() - start
print(f"Operation took {elapsed:.3f}s")

Start with the highest-level operations. If a request takes 2 seconds, time each major phase: database queries, external API calls, computation. Find where the time actually goes.

Profilers

A profiler instruments your code to record how much time is spent in each function. Most languages have tools:

• Python: cProfile, py-spy, line_profiler
• JavaScript/Node: V8 profiler, clinic.js
• Ruby: stackprof, ruby-prof

Database Query Logging

Most databases and ORMs can log queries and execution times. Enable slow query logging to find N+1 patterns, missing indexes, or expensive scans you didn't know were happening.

Browser DevTools

The Network tab shows every request, its size, and duration. The Performance tab gives rendering, scripting, and paint breakdowns with specific recommendations.

AI Cost Monitoring

Track token usage per request type. Most AI APIs return token counts in the response. Log them, aggregate them, and look for outliers — requests using 10x the typical tokens are candidates for prompt optimization.

The Concrete List

Database Mistakes

Code Mistakes

AI Mistakes

Frontend Mistakes

Quick Reference

The Four Root Causes

1. Doing too much work — wrong algorithm, unneeded operations
2. Doing work repeatedly — compute once, cache the result
3. Waiting for things — latency is the enemy, reduce round trips
4. Moving too much data — fetch only what you need

Big O Shapes to Recognize

AI Performance Rules of Thumb

Finding Bottlenecks

1. Time the high-level operations first
2. Find the big one
3. Profile or add logging inside it
4. Fix the actual bottleneck, not what you guessed

What You Can Do Now

Pick one system you're building or using — a web app, an AI workflow, a script that processes data. Ask three questions:

Where does time go? Add basic timing around your main operations. Is it the database? An API call? A computation? Most people don't actually know until they measure.

How much data am I moving? In your database queries, how many rows? Are you selecting all columns? In your AI prompts, how many tokens — and do you need all of them?

Am I doing work repeatedly? Is any expensive operation called more than once with the same inputs? Is any network call happening in a loop?

These three questions surface most real-world performance problems. The fix often takes less time than finding the problem.