AI Product Manager Course Full Index · LLM Fundamentals & AI Harness | xueai.miyang.cn

Full Course

AI Fundamentals
to Mastery

Two parts: from LLM fundamentals to AI Harness — helping AI product managers build a complete mental model and hands-on skill set.

Part 1 · LLM Fundamentals

1

Token & Training Principles

2

GPT Leap · From Completion to Conversation

3

Hallucination Causes & 4 Mitigation Strategies

Part 2 · AI Harness

Context + Prompt Engineering →

Agent Design & Security Defense →

5-Layer Cost Optimization →

Instructor Note

Click any page on the right to jump to it, or click "Start Course" to play through in order.
Each page reveals naturally as you scroll; "Next Lesson" at the bottom continues the sequence.

Introduction

Where Are We? The Dunning-Kruger Effect

Use the Dunning-Kruger curve to locate where learners currently stand and clarify the course goal: from peak of "Mount Stupid" to the Plateau of Sustainability

Why Spend Time on Fundamentals?

Every AI Harness operation is fundamentally about manipulating the message list. Understand this, and every solution becomes transparent

Fundamentals

AI's Food: Training Data

What does 15T Tokens actually mean? Corpus composition visualization + data scale intuition slider

Training vs. Inference: Two Completely Different Processes

Conversations are not learning; parameters are frozen; billing is per Token — the underlying logic every AI PM must know

Vocabulary & Training

From corpus to co-occurrence matrix — tokenization + attention weight interactive demo

Base Model: The Token-by-Token Prediction Machine

What do you get after training? Step-by-step generation + live probability distribution updates

GPT's Leap: PreTraining Changes Everything

Interactive comparison of CNN / RNN / BERT / GPT, with memory-decay visualization

From Completion to Conversation

The Mystery of chat/completions

It's a conversation, so why is the API called "completions"? Explained with a typewriter animation

Faking a Chat History

OpenAI's original experiment: turning a completion machine into a chatbot

Chat Template + SFT

Jinja format, instruction fine-tuning — large models finally learn to "talk"

Why Prompts Are Enough

The Context Window Is the Key

Why are prompts enough? Token truncation visualization — no retraining needed

Hallucination & Mitigation

LLM Hallucination Demo

Three typical hallucination types: factual error / confident fabrication / knowledge cutoff

Mitigation 1: Prompt Engineering

Constraint instructions + limitations: the model doesn't know what it doesn't know

Incl. Limitations

Mitigation 2: RAG – Retrieval-Augmented Generation

Inject real documents into context, 5-step interactive flow animation, with vs. without RAG comparison

Incl. Limitations

The Cost of RAG & Optimization Strategies

Cost analysis table + 4 strategies: keyword trigger / model routing / semantic cache / precision chunking

Mitigation 3: Temperature & Top-P

Drag the slider to see probability distributions and output changes in real time

Incl. Limitations

Mitigation 4: Evaluation + Human Review (HITL)

External error-correction layer, fallback strategy for the cold-start phase

Incl. Limitations

Part 1 Summary (A) · What Are LLMs + Hallucination

Training essence / Token / Base→SFT→Chat / four hallucination types and root causes

Part 1 Summary (B) · Mitigation Strategies + AI PM Decision Framework

Four mitigation strategy comparison / common misconceptions checklist / solution selection decision matrix

──── Part 2: AI Harness ────

Context Engineering

Context Window: AI's Working Memory

Window composition visualization, drag to simulate overflow, capacity comparison across major models

Context Overflow: Three Handling Strategies

Hard truncation / summary compression / selective retention — visual pros and cons comparison for each strategy

Prompt Engineering

Why LLMs Choose Markdown

Plain-text model + formatting needs = Markdown is the natural fit — step-by-step reasoning why HTML/Word/LaTeX don't work

Markdown Syntax & Rendering Pipeline

Quick syntax reference + live editor + marked.js / react-markdown / markdown-it rendering options

Tell It What to Be

Five roles switchable in real time, output format control, core principles of System Prompt

Advanced Prompt Techniques

Few-Shot / CoT / constraints / task decomposition — good vs. bad comparison interactive demo

Output Format Trade-offs

Plain text / JSON / Markdown / YAML / XML — scenario suitability comparison, Token cost vs. parsing complexity trade-off

Streaming Output & Format Coordination

JSON waits for full text / MD renders char by char / XML renders on tag capture — live demo of all three differences

Prompt Injection: Why Attacks Succeed

SQL injection analogy → Message List nature → lack of parameterization → overview of 5 major attack types

Security Step-by-Step

Prompt Injection: 12 Attack Case Studies

Privilege escalation / role-play / Few-Shot / structural injection / metaphor disguise — each case: vulnerable version vs. defended version

Security Interactive

Prompt Defense: TiedStory 3-Layer Interception in Practice

Input-layer regex → prompt-layer constraints → output-layer leak detection → frontend + secondary review — full attack chain simulation

Hands-On Interactive

Agent Engineering

Agent: AI That Gets Things Done

Four core capabilities: Plan / Tool / Memory / Act — click to see real-world cases

The Secret of Tool Calling

Model outputs JSON → framework parses and executes → result injected back — four-step flow visualization

5 Messages Behind One Conversation

Breaking down the real Function Calling chain: user sees 1 reply, but 5 API messages happen behind the scenes

Interactive NEW

The Art of Tool Descriptions

Same functionality, good vs. bad description: success rate difference of 3×, comparative experiment

Interactive NEW

Multi-Tool Orchestration: Concurrent vs. Sequential

isConcurrencySafe determines whether tools can run in parallel — scheduling strategy visualization

Interactive NEW

MCP Protocol: The USB Port for Tools

Comparing stdio / SSE / Streamable HTTP transport modes, with data-flow animation

Interactive NEW

ReAct in Practice: Full Weather-Query Chain

Step-by-step Thought / Action / Observation demo — complete 7-step Agent chain

Short-Term Memory = Context Window

Message list visualization — Token cost and role tag for each message

Interactive NEW

Context Compression: Four Defense Layers

60% trim → 75% micro-compress → 85% fold → 95% emergency — drag the slider to watch the compression

Long-Term Memory: Vector Retrieval

Embedding → vector database → semantic search — design decisions for topK and minScore

Interactive NEW

ReAct Loop: Think → Act → Observe

A console.log removal task goes through 14 loop iterations, including self-correction

5 Ways an Agent Gets Stuck

Parameter format errors, hallucinated tools, infinite recursion, insufficient information, API exceptions

Interactive NEW

Permissions & Security

5 permission modes + LLM risk tiering + Human-in-the-loop design

Interactive NEW

Skill: Keeping Agents on the Right Path

Skill = process instructions + tool call guidance — "balcony laundry" analogy for good loop vs. bad loop

The Essence of a Skill

Good loop vs. bad loop (upgraded) — how a Skill changes an Agent's execution path

Interactive NEW

Dissecting a Real Skill

Learn the structural design of SKILL.md from Alice's source code

Interactive NEW

Scaffolding Engineering: Agent from Prototype to Product

Simulate an Agent searching flights and booking hotels — full comparison: no scaffolding vs. with scaffolding, 5 key capabilities explained

5 Engineering Guardrails

Iteration cap, output truncation, timeout control, interrupt recovery, context emergency

Interactive NEW

Multi-Agent Collaboration

Sub-Agent scheduling, Worker Thread, parallel vs. sequential execution strategies

Interactive NEW

Event stream visualization, Token tracking, OpenTelemetry integration

Interactive NEW

Agent Engineering Full Panorama

From four core capabilities to production deployment — one page, the complete Agent knowledge map

Cost Optimization

Why Does Multi-Turn Conversation Keep Getting More Expensive?

Token cumulative cost visualization — drag the turn count to see how costs grow exponentially

KV Cache: Trading Space for Time (and Money)

Analogy explainer + savings calculator — drag turns to see savings ratio

Explicit Caching: Anthropic vs. Qwen Hands-On Comparison

cache_control syntax, cache hit detection, price discount comparison — real savings demo

Dynamic Timestamps: The Most Expensive System Prompt Design

Wrong design vs. correct design — compare and switch between three time-handling approaches

Holistic Cost Optimization: Saving Money at the System Level

5-layer optimization strategy, cost structure visualization, system design that saves 70-90%

Image Tokens: Pixels Burn Money Too

Image billing formula, scaling mechanism, resolution traps, task-based tiering strategy

Match Resolution to Task: Fine-Grained Image Cost Control

High / medium / low resolution strategies — Token cost comparison and model selection advice for different scenarios

Syntax-Layer Optimization: Prompts Written for Machines

YAML vs JSON, CSV vs arrays, compressed JSON output — save 10-30% on formatting Tokens

Prompt Engineering

Semantic-Layer Optimization: Don't Use Context as a Dump

Dynamic Few-Shot, LLMLingua-2 compression, put key info first and last — improve information density

Output Layer + Advanced KV Cache

Negative constraints, diff-based polish, stop sequences — KV Cache tool traps and sliding-window issues

Model Selection: Capability vs. Cost Trade-offs & Decision Framework

Major model capability/cost matrix, selection decision tree, model matching strategies for different scenarios

The Simple Truth: Hold to First Principles

The essence of AI Harness / the standard for doing vs. not doing / which Harness approaches will be obsoleted / the ultimate question

Closing Thoughts

Part 2 Summary (A) · Prompt Engineering + Agent Design

Context overflow strategies / 6 Prompt elements / the truth about tool calling / Skill + scaffolding

Part 2 Summary (B) · Cost Optimization + Complete AI PM Perspective

Five-layer cost framework / KV Cache fundamentals / Image Tokens / complete course capability checklist