Generative AI integration covers LLMs, embeddings, retrieval-augmented generation (RAG), agent frameworks, and the operational concerns of cost, latency, and prompt governance.
9.2 Generative AI Integration
Three integration patterns
Pattern
Behaviour
Direct API
Prompt to model; immediate answer
Retrieval-Augmented
Retrieve relevant docs first; ground the prompt
Agentic
Model uses tools (search, db, API) to complete
Retrieval-Augmented Generation (RAG) flow
Key model families
Family
Examples
Commercial
GPT (OpenAI), Claude (Anthropic), Gemini
Open weights
Llama, Mistral, Qwen, Gemma
Image
Stable Diffusion, FLUX
Speech
Whisper, gpt-realtime
Operational concerns
Concern
Practice
Cost
Cache aggressively; track input+output tokens
Latency
Streaming for perceived performance
Privacy
No PII in third-party prompts unsafe
Hallucination
Ground every assertion; require citations
Versioning
Lock model versions; test before upgrade
Evaluation
Golden set + regression + qualitative review
Risks for the enterprise
Risk
Mitigation
Confidential data leak
Prompt PII filter + private models
Copyright exposure
Provenance tagging on generated content
Authority bias
UX states 'AI-generated; verify'
Cost surprises
Per-team quota + usage alerting
Vendor lock-in
Abstract via gateway; portable prompts
Mentor’s tip: GenAI = direct API + RAG + agents. Treat retrieved chunks as primary; never let the model invent facts. Token cost, prompt PII, and silent model upgrades are the three operational traps to control.
Discussion
Loading…