Quiz

# ============================================================
# Q1) When should you choose LCEL over an agent?
# ============================================================
# ANSWER: when the workflow is deterministic.
# LCEL pipes are simple, debuggable, and cheap. Agents (langgraph included)
# are the right call only when the model has to DECIDE which tool to use.
# Most 'AI features' are extraction or summarisation — LCEL wins those.

# ============================================================
# Q2) Which retriever for a corpus of 1M documents with metadata filters?
# ============================================================
# ANSWER: a managed vector DB with hybrid search.
#   Pinecone / Weaviate / Vespa / Qdrant Cloud
# Add BM25 or sparse-vector hybrid for keyword recall.
# Avoid in-memory Chroma at that size — it does not scale operationally.

# ============================================================
# Q3) Which output strategy for structured extraction?
# ============================================================
# ANSWER: llm.with_structured_output(PydanticModel)
# Built-in, type-safe, handles retries on validation failure.
# Reach for Guardrails / Instructor only when you need custom retry policies
# or runtime schema editing.

# ============================================================
# Q4) Where should memory live?
# ============================================================
# ANSWER: never on the global LLM object. Per-request, per-user.
# Use a session id, persist turns to a DB, hydrate before each call.
# A module-level memory in a serverless function is a cross-tenant leak.

# ============================================================
# Q5) What is the most cost-effective way to use GPT-4-class models?
# ============================================================
# ANSWER: route by intent.
#   Default to a small model (gpt-4o-mini, Haiku).
#   Use a router (rule-based or a tiny classifier) to escalate ONLY the queries
#   that need the big model.
# 80/20 rule: 80% of queries are easy and go to the cheap path.

# ============================================================
# Q6) When do you need DataLoader-style batching with LLMs?
# ============================================================
# ANSWER: when you process LISTS in a batch job (e.g. tag every product).
# Wrap calls in asyncio.gather() with a concurrency limit (Semaphore).
# Use the providers Batch API where it exists for 50%+ discounts and
# multi-hour SLAs.

# ============================================================
# Q7) How do you evaluate a chain BEFORE deploying?
# ============================================================
# ANSWER: a golden dataset + cheap evaluators + a regression gate.
#   1. 50-200 representative inputs with ground-truth or rubrics
#   2. Cheap evaluators first (exact match, contains, length, schema valid)
#   3. LLM-as-judge sparingly for nuance
#   4. Gate CI on regressions vs the last commit
# This is a small investment that catches 'cleaning up a prompt broke 12% of cases'.

# ============================================================
# Q8) Prompt-injection: what is the right defence layer?
# ============================================================
# ANSWER: assume injection WILL happen. Defend with structure.
# - Validate output against a schema; reject malformed
# - Scope tool access (least-privileged)
# - Filter known phrases (heuristic, not a wall)
# - Never run shell, eval, or DB writes on output text directly
# - Strip / redact PII before sending to the model
# Treat the LLM as untrusted code. Sandbox it.

# ============================================================
# Scoring
# 8 / 8 -> can lead a LangChain design review
# 5 / 8 -> bookmark the langchain/cheatsheet lesson
# < 5   -> read the prod-grade-langchain docs section before shipping

# 3 questions per lesson.