AWS Certified Generative AI Developer – Professional (AIP-C01)

Exam Study Notes & Prep Guidance

These are my personal study notes for the AWS Certified Generative AI Developer – Professional (AIP-C01) exam.
I passed the exam and earned Early Adopter status.

This exam is closer to Pro-level certifications in expectations and mindset, but with less technical depth than other AWS Pro exams. It is by far easier than the AWS Certified Advanced Networking – Specialty, but still legitimately challenging due to gaps in available training material.

The difficulty comes less from memorization and more from reasoning through GenAI architecture, tradeoffs, cost, security, and operational scenarios.

Recommended Prerequisites (Strongly Suggested)

Before attempting AIP-C01, you should already be comfortable with:

• AWS Certified AI Practitioner (AIF-C01)
• AWS Certified Solutions Architect – Associate (SAA-C03)

These should realistically be considered prerequisites, not optional prep.
Having other AWS Professional-level certifications helps significantly, especially for architecture and security-related questions.

Exam Style & Expectations

Expect many questions framed around tradeoff analysis rather than raw service knowledge, often using wording such as:

• “Least operationally expensive”
• “Most cost-effective”
• “Simplest to operate at scale”
• “Minimize ongoing maintenance”

You are frequently asked to choose between multiple technically valid solutions, where the correct answer depends on operational burden, cost, security posture, and long-term maintainability — not just whether something works.

This exam assumes you already have a strong, holistic understanding of AWS, well beyond GenAI-specific services. You are expected to reason confidently about:

• IAM (roles, policies, trust relationships)
• Service Control Policies (SCPs)
• AWS Config and governance controls
• Networking & security boundaries
• Database options and tradeoffs
• Observability (CloudWatch, logging, metrics, alarms)
• Cost management and operational overhead

Because of this, having taken at least one AWS Professional-level exam beforehand is extremely helpful. The GenAI Developer – Professional exam builds on that architectural and operational mindset rather than teaching it from scratch.

How to Prepare (High-Level Strategy)

The most effective approach I found:

1. Learn the material
   • Use AWS Skill Builder and/or Udemy to understand the concepts and services
2. Practice exam-style reasoning
   • Go through official practice questions (Skill Builder currently has the best-quality questions)
3. Use AI as a study partner
   • Break down questions
   • Explain why answers are right or wrong
   • Identify architectural patterns and traps

Useful Links

Official AWS resources

• Official Exam Guide – AWS Certified Generative AI Developer – Professional (AIP-C01)
  https://d1.awsstatic.com/onedam/marketing-channels/website/aws/en_US/certification/approved/pdfs/docs-aip/AWS-Certified-Generative-AI-Developer-Pro_Exam-Guide.pdf

AWS Skill Builder (official practice content)

• Official Bonus Questions (25 questions, via BenchPrep)
  https://awscertificationpractice.benchprep.com/app/official-bonus-questions-aws-certified-generative-ai-developer-professional-aip-c01#exams

• Official Practice Question Set (20 questions, via BenchPrep)
  https://awscertificationpractice.benchprep.com/app/official-practice-question-set-aws-certified-generative-ai-developer-professional-aip-c01?locale=en-us#exams

• Official Pretest (75 questions) - SkillBuilder Subscription needed !!
  https://skillbuilder.aws/learn/24FDAZ9UKG/official-pretest-aws-certified--generative-ai-developer--professional-aipc01--english/

• Domain Walkthrough Question Sets (Domains 1–5)
  10 questions total (2 per domain); good step-by-step walkthroughs of exam-style reasoning

Third-party prep

• Ultimate AWS Certified Generative AI Developer – Professional (Udemy, Frank Kane + Stéphane Maarek)
  https://www.udemy.com/course/ultimate-aws-certified-generative-ai-developer-professional
  Notes: cleaner organization than Skill Builder; watch at 1.25×+ for efficiency, includes 75 practice questions

General AI concepts

• Foundation Model (FM) – Large pre-trained model provided by AWS or partners, designed to be adapted for multiple downstream use cases.
• Fine-Tuning – Customizing a foundation model by updating model weights using labelled, task-specific training data.
• Continued Pre-Training (CPT) – Adapting a foundation model using unlabelled domain-specific data to extend knowledge without explicit labels.
• Low-Rank Adaptation (LoRA) – Parameter-efficient fine-tuning technique that adds trainable low-rank layers while keeping the base model frozen.
• Retrieval-Augmented Generation (RAG) – Architecture pattern that retrieves relevant data at inference time and injects it into the prompt to ground model responses.
• Embeddings – Numerical vector representations of data that capture semantic meaning for similarity search and retrieval.
• Inference – The process of invoking a trained model to generate predictions or outputs.
• Prompt – Input text and instructions provided to a foundation model to influence its response.
• Prompt Template – Reusable prompt structure with placeholders that standardizes inputs across requests.
• Context Window – Maximum number of tokens a model can process in a single request, including input, retrieved context, and output.
• Tokens – Units of text processed by a model; cost, latency, and limits scale with token usage.
• Temperature – Sampling parameter that controls response randomness; lower values produce more deterministic outputs.
• Top_p (Nucleus Sampling) – Sampling parameter that limits token selection to the smallest set of tokens whose cumulative probability exceeds a threshold.
• Hallucination – Model output that appears coherent but is not supported by training data or provided context.
• Grounding – Technique that constrains model responses to retrieved or supplied data sources to reduce hallucinations.
• Guardrails – Policy-based controls applied during model invocation to enforce safety, compliance, and content constraints.
• Human-in-the-Loop (HITL) – Design pattern where human reviewers validate or correct model outputs for quality or compliance.
• Batch Inference – Asynchronous processing of large volumes of requests optimized for throughput and cost efficiency.
• Multi-modal Model – Foundation model capable of processing or generating multiple data modalities (e.g., text and images).
• Vector Store – Storage system optimized for managing and querying embeddings using similarity search algorithms.
• Semantic Search – Retrieval technique that returns results based on semantic similarity rather than exact keyword matching.

AWS AI & ML services

Language & text

• Amazon Comprehend – Natural language processing: sentiment, entity/key‑phrase extraction, topic modelling and custom entities; includes Comprehend Medical for clinical texts. Custom Classification oragnized documents into user-defined categories.
• Amazon Kendra – Enterprise document search with connectors; can be used as a retrieval layer in RAG architectures.
• Amazon Lex – Conversational interface (chatbot) service.
• Amazon Q – Generative AI assistant: Q Business (with data connectors, plugins, Q Apps) and Q Developer for code assistance.
• Amazon Textract – OCR plus extraction of structured data from documents.
• Amazon Transcribe – Speech‑to‑text. Can be improved with custom vocabularies (domain-specific words) and custom language models (domain specific context).

Vision & multimodal

• Amazon Rekognition – Image/video analysis with Custom Labels and Custom Moderation for content safety.

Safety & governance

• Amazon Macie – Detects and classifies PII and sensitive data in S3.
• Amazon Augmented AI (A2I) – Human‑in‑the‑loop review of machine learning predictions. Use for high‑risk GenAI outputs.

Other AI/ML helpers

• SageMaker family – See dedicated section below.
• AWS Glue – Data integration: Crawlers, Data Catalog, Studio and Data Quality. For GenAI pipelines it can extract, transform and load (ETL) data before embedding.
• AppFlow – Managed SaaS data transfer (e.g., Salesforce to AWS).

Amazon Bedrock

Bedrock is AWS’s managed platform for foundation models and GenAI tools.

Core Bedrock services

• Model catalog – Access to multiple foundation models (Titan, Claude, Llama, etc.). Understand model families and when to choose each.
• APIs – Completions, embeddings, agents and flows.
• Knowledge Bases (KB) – Fully managed RAG pipeline: ingestion, chunking, embedding, storage and retrieval. Supports OpenSearch Serverless, Aurora PostgreSQL (pgvector), Neptune Analytics and S3 Vectors as backing stores.
• Agents – Managed systems that call APIs/tools in response to prompts. You define action groups for each tool. Use Bedrock Agent Tracing and Agent Observability via CloudWatch for debugging.
• Data Automation (BDA) – Extracts structured data from unstructured sources via blueprints.
• Batch Inference – Submit multiple prompts via S3 and retrieve outputs asynchronously.
• Cross‑Region Inference – Distribute inference across multiple regions.
• Intelligent Prompt Routing – Routes requests to different models based on complexity to optimise cost and performance. orchestration).
• Model/Agent Evaluations – Evaluate model quality using metrics or custom datasets.
• Bedrock Flows – Visual pipeline orchestration connecting FMs with data sources/tools.

Rules of Thumb

• Multi-Region Failover → Bedrock cross-Region inference (not traditional Route-53 approach)
• Multi-Region performance routing → inference profiles
• Too many requests and must keep the same model with minimal ops → Bedrock cross-Region inference
• Throttling exceptions -> Provisioned Capacity
• Dynamic model selection by request complexity → Intelligent Prompt Routing
• Scanned or image-based documents processing → Bedrock Data Automation
• Avoid custom OCR / parsing → BDA blueprints
• Multimodal ingestion before RAG → BDA → Knowledge Base
• Confluence / SaaS docs as source → Bedrock Knowledge Base managed connector
• Automatic re-sync on updates → Knowledge Base ingestion jobs
• Large volumes of prompts or documents or embedding to do → Bedrock Batch Inference (optimize cost and throughput by processing requests in bulk instead of per-request inference)

Amazon Bedrock API calls

• InvokeModel – Core synchronous inference API; use for standard, low-latency requests.
• InvokeModelWithResponseStream – Streaming inference; use for real-time token streaming (chat/UX scenarios).
• StartBatchInferenceJob – Asynchronous, large-scale inference from S3; use when you see millions of records, throttling, or idle compute.
• RetrieveAndGenerate – Managed RAG API that performs retrieval + generation in one call; grounds responses in documents to reduce hallucinations (preferred when custom RAG orchestration isn’t required).
• Retrieve – Retrieval-only operation; use when evaluating or debugging retrieval quality independently of generation.
• CreateKnowledgeBase / UpdateKnowledgeBase – Manage Bedrock Knowledge Bases (data sources, vector stores).
• CountTokens – Returns token count without running inference; used for cost estimation and budgeting.
• CreateGuardrail / ApplyGuardrail – Define and enforce policy-based safety controls on inputs/outputs.
• CreateModelEvaluationJob – Run automated evaluations against datasets for model comparison and regression testing.
• PutModelInvocationLoggingConfiguration – Enable prompt/response logging for auditability and debugging.
• ListFoundationModels / GetFoundationModel – Discover available models and their capabilities (text, embeddings, multimodal).

Rules of Thumb

• Interactive UX → InvokeModelWithResponseStream
• High-volume/offline → StartBatchInferenceJob
• Hallucination reduction, no custom RAG → RetrieveAndGenerate
• Cost estimation → CountTokens
• Latest content grounding → RetrieveAndGenerate

Bedrock Guardrails & safety

• Content filters – Block harmful content categories (includes prompt attacks, violence, hate speech etc.).
• Sensitive‑info filters – Detect and redact PII and other sensitive data.
• Denied topics – Block responses on policy‑defined topics.
• Word filters – Custom blocklists or regex patterns.
• Contextual grounding checks – Ensure answers are grounded in retrieved documents to reduce hallucinations.
• Automated reasoning checks – Enforce logical constraints or policies.
• Tiers – Standard tiers provide improved robustness (typo tolerance, multi‑language support).

Agents & AgentCore

• Bedrock Agents – Provide orchestrated reasoning and tool invocation. Use action groups to define accessible APIs.
• Bedrock AgentCore – Managed runtime to deploy agents at scale; works with any agent framework (including Strands Agents). Includes AgentCore Gateway for scalable access to external APIs/tools.

Bedrock Knowledge Base vs. custom RAG

• Use Knowledge Bases when you want a fully managed RAG solution with minimal code. AWS manages ingestion, embedding and retrieval across supported vector stores.
• Build custom RAG when you need control over chunking, embeddings or storage. You might integrate OpenSearch, Aurora pgvector, Neptune, S3 Vectors, or third‑party vector stores.

Multi‑agent systems & patterns

• Orchestrator – Breaks down tasks and delegates to specialised agents.
• Router – Routes work to appropriate specialised agents.
• Synthesiser – Merges outputs from multiple agents.
• Prompt chaining – Sequence of LLM calls with intermediate prompts; may include gates (conditional paths).
• Evaluator/optimizer – One model grades or improves another model’s output.

Strands Agents vs. AWS Agent Squad

• Strands Agents – Lightweight framework for experimentation or custom logic; you manage orchestration and scaling. Good for prototypes or local workflows.
• AWS Agent Squad – Managed multi‑agent orchestration for production workloads with governance and scaling. Integrates tightly with Bedrock and AgentCore. Use when you need secure, auditable, production‑scale agent workflows.

Model selection & generation parameters

Choosing the right model family is primarily about modality, output type, and operational simplicity.

Model types

• Text (generation) models – Text in → text out; use for chat, summarization, reasoning, and code.
  • Examples: Amazon Titan Text, Anthropic Claude
• Embedding models – Input → vector embeddings; use for semantic search, RAG, clustering.
  • Examples: Amazon Titan Embeddings (text-only), Titan Multimodal Embeddings
• Multi-modal models – Handle multiple modalities (text + images); use for cross-modal understanding.
  • Examples: Titan Multimodal Embeddings, multimodal Claude variants (vision-capable)

Choosing a model

• Pure text generation → Text model (e.g., Claude, Titan Text)
• RAG / semantic search → Embedding model + separate text generation model
• Images + text, single vector space required → Multimodal embedding model
• Explain images in natural language → Multimodal text model (vision-capable Claude)
• Minimize system complexity → Prefer a single model that satisfies all modalities

Evaluating model outputs (AWS exam-aligned)

• Perplexity – Measures how well a model predicts the next token; use for training or fine-tuning evaluation, not output correctness.
• BLEU – N-gram precision metric; use for machine translation against reference text.
• ROUGE – Recall-focused overlap metric; use for summarization quality.
• BERTScore – Embedding-based semantic similarity; use for meaning preservation in free-form text.

Rules of thumb:

• Model training quality → Perplexity
• Translation → BLEU
• Summarization → ROUGE
• Semantic / open-ended text quality → BERTScore

Exam note: Perplexity does not measure hallucinations or grounding; use task-specific metrics or human review for GenAI apps.

Model Output Tuning

• Controlled but varied responses → temperature ~0.4–0.6, top-p ~0.7–0.9
• Highly deterministic / repeatable output → temperature ≤0.2, low top-p
• Some variation without hallucination risk → moderate temperature + moderate top-p
• Creative / exploratory generation → temperature ≥0.8, high top-k or top-p
• Strict response length requirement → response length limits or penalties
• Prevent rambling → length penalties, not stop sequences
• Safety- or policy-constrained output → keep temperature moderate, don’t rely on stop tokens

Bedrock observability - Which feature answers which debugging question?

• PreProcessingTrace → What exactly did the agent receive and how was it interpreted? (Detect prompt injection, malformed input, bad normalization)
• OrchestrationTrace → Why did the agent choose this plan / tool / step order? (Debug reasoning paths, branching logic, hallucination root causes)
• PostProcessingTrace → How did the final answer get shaped or filtered? (Formatting issues, redactions, guardrail side effects)
• FailureTrace → Where and why did the agent fail? (API errors, tool timeouts, retries, broken steps)
• GuardrailTrace → What safety rule blocked or modified the response? (PII, toxic content, denied topics)
• ModelInvocationInput / Output Trace → What did the model actually see and return? (Prompt quality, grounding issues, unexpected completions)
• CloudWatch metrics (tokens, latency, errors) → Is the system healthy and scalable? (Throughput, throttling, cost, performance — not reasoning)
• Golden dataset comparison → Is behavior drifting (quality drift) or hallucinating over time? (Regression detection, quality validation)

ReAct vs Agents vs Flows

Aspect	ReAct (Step Functions)	Bedrock Agents	Bedrock Flows
What it is	Explicit state-machine reasoning	Model-driven tool use	Visual orchestration
Who controls flow	You (code / states)	Model	You (diagram)
Reasoning visibility	High (per-step outputs)	Medium (agent traces)	Medium
Branching	Deterministic	Implicit	Explicit (limited)
Auditability	High	Medium	Medium
Best fit	Regulated, high-risk decisions	Conversational assistants	Simple pipelines
Determinism	High	Medium	Medium
Typical exam use	Investigations, compliance	Chatbots, helpers	Low-code workflows

Quick decision guide

• Need auditability or guarantees → ReAct
• Need autonomy and flexibility → Agents
• Need visual, low-code orchestration → Flows

Quality & Safety Gates in a Production GenAI Pipeline (Training vs Inference)

Layer question	Applies to	What it protects against	Typical problems	Common AWS tools
Is the data structurally sane?	Training + Inference	Garbage input	Empty records, missing fields, unsupported values, schema drift	AWS Glue Data Quality, AWS Glue ETL, AWS Glue Data Catalog
Is the data safe to use?	Training + Inference	Sensitive or malformed input	PII, PHI, mixed languages, disfluent text	Amazon Comprehend (PII + language detection), AWS Lambda (normalization/masking), Amazon Transcribe (speaker labels, language ID)
Is the output safe to return?	Inference only	Harmful or non-compliant output	Toxic content, policy violations, leakage	Amazon Bedrock Guardrails, Bedrock content filters
Is the output correct and useful?	Inference (primary)	Wrong or low-quality answers	Hallucinations, irrelevance, inconsistency	Amazon Bedrock Knowledge Bases, Amazon OpenSearch (vector search), metadata filtering, prompt templates, temperature / top_p

PII Detection on AWS — When to Use What

Service	Where it runs in the pipeline	Applies to	What it’s best at	Typical use cases	When it’s the WRONG choice
Amazon Bedrock Guardrails	At model invocation (input + output)	Inference only	Preventing PII from reaching or leaving the model	Redact/mask PII in prompts and responses; enforce privacy with minimal code; ensure PII is never returned	Cleaning historical data; batch processing S3 objects; non-GenAI workloads
Amazon Comprehend	Before the model (data preprocessing)	Training + Inference	Detecting and transforming PII in raw text	Redact PII in transcripts or documents; normalize text before RAG; language detection + entity extraction	Real-time GenAI enforcement; output filtering; zero-code pipelines
Amazon Macie	After storage (S3 scanning)	Training data / at rest	Discovering sensitive data at rest	Find where PII exists in S3; compliance audits; security posture visibility	Preventing storage of PII; redaction or transformation; inline application flows

Rules of Thumb

• Guardrails alone ≠ jailbreak defense → Add pre-model classifiers
• Detect jailbreak intent, not just keywords → Bedrock safety-classifier
• Block before the model sees the prompt → Lambda pre-processor
• Defense-in-depth for GenAI → Pre-filter + Guardrails + Monitoring
• Schema validation + completeness checks → AWS Glue Data Quality
• Dataset-level validation (not per-record logic) → AWS Glue ETL + Data Quality rules
• Minimize Code Changes → Likley NOT Lamnda but Guardrails instead

Rules of Thumb — Agent, Model & RAG Evaluation and Performance

Core evaluation selection

• Compare multiple foundation models on the same task → Bedrock Model Evaluations
• Evaluate RAG end-to-end (retrieval + answer quality) → RAG evaluation (retrieve-and-generate)
• Evaluate retrieval quality only → RAG evaluation (retrieve-only)
• Measure correctness, completeness, faithfulness, coherence → Bedrock evaluation jobs (LLM-as-judge)
• Have ground-truth answers and ideal contexts → Provide reference answers + reference contexts in S3
• Minimize custom evaluation infrastructure → Use Bedrock evaluation jobs

Agent-specific evaluation

• Evaluate agent tool selection, reasoning flow, and final output → Bedrock Agent Evaluations
• Validate agent behavior across scenarios (happy path + edge cases) → Agent evaluations with predefined prompts
• Compare agent versions or configurations → Agent evaluations (same inputs, different configs)

RAG-specific rules

• Unsure whether errors come from retrieval or generation → Run retrieve-only evaluation first
• “Is the model using the right documents?” → Retrieve-only RAG evaluation
• “Is the final answer correct and grounded?” → Retrieve-and-generate RAG evaluation
• Need citation coverage or document faithfulness metrics → RAG evaluation with reference contexts
• Tuning chunking, filters, metadata, or index settings → Retrieve-only evaluation before prompt or model changes

Dataset & workflow clues

• Prompt or evaluation dataset already in S3 → Bedrock evaluation jobs
• Pre-production model or agent bake-off → Model Evaluations
• Repeatable, automated scoring required → Evaluation jobs (not ad-hoc scripts)
• LLM-as-judge explicitly mentioned → Bedrock evaluation jobs (by definition)

What NOT to use for quality evaluation

• Latency, token count, error rate ≠ model quality → Do not use CloudWatch metrics
• User feedback alone ≠ ground truth → Not sufficient for model comparison
• Manual review ≠ scalable evaluation → Fails automation and repeatability
• Operational monitoring ≠ evaluation → CloudWatch is for ops, not correctness

Supporting services (where they fit)

• CloudWatch → Operational health (latency, errors, throttling)
• CloudWatch Synthetics → Endpoint availability and basic response checks (not GenAI quality)
• Bedrock Guardrails → Safety enforcement, not quality scoring
• SageMaker Clarify → Bias detection (for training data) and explainability (classification/regression models, not LLM text quality)
• Amazon Augmented AI (A2I) → Human review for low-confidence or high-risk outputs (quality control, not automated evaluation)

Fast mental mapping

• Ops health → CloudWatch
• Endpoint up/down checks → CloudWatch Synthetics
• Safety & compliance → Guardrails
• Retrieval quality → RAG eval (retrieve-only)
• Answer quality & grounding → RAG eval (retrieve-and-generate)
• Model or agent comparison → Model / Agent Evaluations
• Bias & explainability (non-LLM) → SageMaker Clarify
• Bias in model outputs (inference / generated text) → BOLD
• User sentiment analysis → Amazon Comprehend
• Human review loops → Amazon A2I
• GenAI runtime visibility → CloudWatch Generative AI observability
• Per-user tracking, cost attribution, or traffic analysis → requestMetadata + CloudWatch Logs Insights

Memory hooks

• Quality ≠ latency → Use evaluation jobs
• CloudWatch tells you how fast; Bedrock eval tells you how right
• RAG problems require RAG evaluation modes
• Agents need agent-specific evaluations, not just model evals

Amazon SageMaker family

• Data Wrangler – Visual data preparation.
• JumpStart – Pre‑built models and algorithms; one‑click deployment.
• Feature Store – Centralised feature repository.
• Ground Truth / Ground Truth Plus – Data labelling (Plus = fully managed).
• Model Monitor – Detects data drift and bias.
• Clarify – Explains model predictions and detects bias.
• Model Registry – Stores and versions models for deployment.
• ML Lineage Tracking – Tracks datasets, code and models across experiments.
• Neo – Train once, deploy anywhere (edge devices).
• Unified Studio – End‑to‑end ML IDE.
• Pipelines – Declarative ML workflow orchestration.
• MLflow on SageMaker – Experiment tracking integration.

Note: Use SageMaker when you need full control over training or hosting models in a VPC. Use Bedrock when you want managed foundation models and serverless inference. In the exam, be ready to choose between these options based on requirements such as control vs. convenience, data privacy, cost and supported frameworks.

DJL (Deep Java Library)

• Used for: High-throughput LLM inference on SageMaker (multi-GPU)
• Key knobs: Continuous batching, tensor parallelism, replicas
• Utilization fix:
  • Prompts much shorter than max → lower max sequence length
  • Model fits on fewer GPUs → reduce tensor parallelism, increase replicas
• Not for: Training, fine-tuning, evaluation Memory rule:

Tune parallelism + sequence length before adding instances.

AWS Glue (data prep)

• Crawlers – Discover and infer schema from data sources.
• Data Catalog – Central metadata store for tables and partitions.
• Glue Studio – Visual ETL development environment.
• Data Quality – Rule‑based quality checks and profiling.

Glue can appear in scenarios for ETL preceding embeddings or fine‑tuning.

Glue vs Lake Formation

• AWS Glue Data Catalog
  • Metadata, discovery, lineage, table registration
  • Answers: “What data exists?”, “Where did it come from?”
• AWS Lake Formation
  • Fine-grained data access enforcement (row/column-level)
  • Answers: “Who can query which columns/rows?”

Rule of thumb

• Knowing what data exists* → Glue
• Controlling who can access it → Lake Formation
• S3 Fine grained permissions → Lake Formation

Model Context Protocol (MCP)

• MCP – Standardized protocol that lets LLM agents call external tools safely and consistently.
• Purpose – Decouples agent reasoning from tool implementation; agents speak tools, not REST.
• What MCP standardizes – Tool schemas, inputs, outputs, and invocation semantics (not compute).
• Security & safety – Enables strict argument validation, input constraints, and safer tool execution.
• Deployment model – Each MCP server is deployed independently on compute that matches the tool’s workload.

Design rules (exam-relevant):

• Use one MCP server per tool or closely related toolset for clear boundaries and blast-radius control.
• Put an MCP boundary in front of external or fragile APIs (rate limits, strict schemas, side effects).
• Avoid letting agents directly call raw REST APIs.
• MCP is about interface consistency, not orchestration (that’s Agents / Step Functions).

GenAI Security

Identity & Access

• Enterprise users (AD / Entra ID) → IAM Identity Center + SAML / OIDC
• Department / OU isolation → Permission sets + IAM conditions (bedrock:ModelId)
• Org-wide hard enforcement → SCPs (deny unapproved models regardless of IAM)
• Least privilege → IAM policy conditions > app-layer controls

Network Security

• Private subnet access → VPC Interface Endpoint (PrivateLink)
• Enforce no public internet → SCP or IAM condition requiring VPC endpoint
• Exam trap → never NAT, ALB, or proxy Bedrock for “private-only” access

Model & Content Controls

• Inference-time safety → Bedrock Guardrails (topics, PII, denied content)
• Guardrail tuning & insight → enable guardrail tracing
• Pre-model analysis (optional) → Lambda + Comprehend
• Post-inference workflows → EventBridge + Lambda (not primary control)

Audit & Observability

• Who invoked which model → CloudTrail (Bedrock API calls)
• Why content was blocked → Guardrail tracing + CloudWatch metrics
• Org-wide visibility → central logging once, not per app

Governance Patterns (Exam Favorites)

• Restrict allowed models → SCP with bedrock:ModelId condition
• Cross-account consistency → Identity Center permission sets
• Compliance documentation → model cards (SageMaker Model Registry)
• What not to do → custom auth proxies, per-account IAM users, prompt-only controls

AI data stores & vector databases

OpenSearch

• Search & analytics engine (not OLTP) with vector capabilities.
• Vector search types:
  • Exact nearest neighbour (NN) – High precision, slower.
  • Approximate NN (ANN) – Trade recall for speed. Two key algorithms:
    • HNSW (Hierarchical Navigable Small World) – High recall and low latency; uses more RAM. Good for low‑latency, high‑quality search.
    • IVF (Inverted File) – Good for very large datasets; allows recall‑speed tuning.
• Neural plugin – Built‑in embedding and search pipelines (simplifies RAG).

When to use: Choose HNSW for performance‑critical queries; choose IVF for extremely large datasets or when memory savings are important.

OpenSearch Optimization (Vector & RAG workloads)

• Shard strategy

  • Prefer fewer, larger shards for vector-heavy semantic search
  • Too many shards increase query fan-out and latency

• Hierarchical index design

  • Use a lightweight router index (e.g., product line, topic, tenant)
  • Route queries to one or a few detailed vector indices
  • Reduces search space and cost for ANN queries

• Index-level optimizations

  • Tune HNSW parameters (ef_search, ef_construction) for recall vs latency
  • Separate hot vs cold indices when access patterns differ
  • Use metadata filters to narrow candidate vectors before ANN

• Query patterns

  • Prefer hybrid search (keyword + vector) for better relevance
  • Cache frequent queries upstream when possible

• Natural language queries → Neural search

• Semantic similarity required → Dense vectors

• Exact terms or identifiers matter → Sparse (BM25)

• Mixed technical + natural language content → Sparse + Dense hybrid

• Relevance tuning or scoring mentioned → Hybrid

• If unsure → Hybrid

• If hybrid unavailable → Dense

OpenSearch Neural Plugin

• Use when you want OpenSearch to accept raw text queries and generate embeddings internally (no client-side embedding code).
• Pick when you want OpenSearch ingest/search pipelines to call Bedrock embedding models directly via a connector.
• Good fit when you already operate OpenSearch and want DIY RAG without Bedrock Knowledge Bases.
• Use for custom indexing logic or hybrid search (keyword + vector) tightly coupled to OpenSearch.
• Prefer over Knowledge Bases when you need full control over indices, shard strategy, and query DSL.
• Avoid when you want minimal ops / managed RAG → use Bedrock Knowledge Bases instead.
• Avoid if embeddings are generated elsewhere and stored directly → Neural plugin adds no value.

Rules of thumb

• Managed RAG, minimal plumbing → Bedrock Knowledge Bases
• OpenSearch-centric RAG with text-in / vector-out handled by OpenSearch → Neural Plugin
• Client controls embeddings explicitly → No Neural Plugin

S3 Vectors

• Lowest‑cost vector store; managed via S3. Suitable for large, cold datasets. AWS often recommends combining S3 Vectors for bulk storage with OpenSearch for hot, low‑latency queries.

Aurora pgvector

• Amazon Aurora (PostgreSQL) supports the pgvector extension. Use for small/medium datasets when you need SQL capabilities alongside vector similarity search. Supports HNSW and IVF indices.

ElastiCache & MemoryDB

• ElastiCache (Valkey) – Provides in‑memory vector search for ultra‑low‑latency queries. (more setup needed then ElasticCache)
• MemoryDB – Durable, in‑memory vector store; fully managed and designed for high‑throughput workloads.

DynamoDB

• Not used for vectors but valuable for storing session state, metadata and conversation memory.

DynamoDB Tips (usage for chat history)

• Chat history + scale → DynamoDB
• Resume conversations → conversationId as partition key
• Metadata filtering → GSI
• Hot recent reads → DAX
• Automatic retention → TTL
• Avoid cron deletes → TTL beats scheduled jobs
• If it smells like state, not search → not OpenSearch

Pinecone

• Pinecone – Managed, serverless vector database that automatically scales and offers simple APIs. It integrates with AWS services and Bedrock Knowledge Bases as an external vector store option. Use Pinecone when you need hassle‑free setup, auto‑scaling and multi‑cloud portability; choose AWS‑native stores for tighter integration, lower latency within AWS and potentially lower cost.

MongoDB Atlas (Vector Search)

• MongoDB Atlas Vector Search – Managed vector search built into MongoDB Atlas.
• Supports hybrid use cases: document store + vector search in one system.

Vector store selection summary

• OpenSearch – Best general‑purpose engine for high‑performance RAG.
• S3 Vectors – Cheapest storage for large collections.
• Aurora pgvector – SQL + vectors for moderate datasets.
• MemoryDB – Ultra‑fast, in‑memory search.
• Pinecone – Managed, serverless and auto‑scaling; good for ease of use and cross‑cloud portability.
• MongoDB Atlas – Document DB + vector search in one platform.

RAG Relevance Optimization

• Too many relevant docs, best ones ranked low → Rerankers
• Poor recall with vector-only search → Hybrid search (vector + keyword)
• Want fastest improvement, least infra → Knowledge Bases + OpenSearch + Bedrock rerankers
• Avoid custom ranking logic unless explicitly required

Rules of Thumb

• Default RAG on AWS → Bedrock Knowledge Bases
• Documents already in S3 → S3-backed Knowledge Base
• Minimal ops / no ingestion code → Knowledge Base + StartIngestionJob
• Need metadata filtering → metadata.json with Knowledge Base
• Automatic index sync on S3 changes → S3 event → StartIngestionJob
• Avoid cluster management → OpenSearch Serverless
• DIY pgvector → Only if you need SQL semantics outside RAG
• Search engine + high QPS + strict latency requirements → Amazon OpenSearch (provisioned)
• Need fine-grained relevance tuning (boosts, hybrid scoring, ranking logic) → Amazon OpenSearch (sparse + dense hybrid)
• Managed RAG with minimal infrastructure and glue code → Amazon Bedrock Knowledge Bases
• Enterprise document search with built-in connectors and managed relevance → Amazon Kendra
• Serverless search with lower operational overhead but fewer tuning knobs → Amazon OpenSearch Serverless
• Return snippets, highlights, and document references at scale → Traditional search engine (OpenSearch/Kendra), not pure RAG
• RAG for answer generation, not search ranking → Bedrock Knowledge Bases
• Need to tune relevance independently of the FM → Search layer (OpenSearch/Kendra), not the model

Metadata & filtering:

• Simple metadata filtering → metadata.json in Knowledge Base
• Per-document attributes (tenant, product, region) → metadata.json
• RAG explainability / traceability → propagate metadata into embeddings
• Access control via retrieval filters → metadata-based filtering (not IAM)
• Need complex joins or relational filters → Aurora pgvector (not Knowledge Bases)
• Need per-tenant isolation at index level → separate Knowledge Bases or vector stores

Chunking, embeddings, and vector stores

Core concepts (mental model)

• Underlying data source – Original document (PDF, HTML, DOCX, Confluence page, etc.).
• Chunk – Logical text segment extracted from the source and embedded.
• Vector – Numerical embedding that represents a chunk in the vector store.
• Metadata – Key–value attributes attached to a chunk/vector (e.g., tenant, source, product, section).
• Retrieved chunk – Text returned at query time; may include more surrounding context than the exact vector span.

Chunking strategies (high-yield)

• Default chunking – ~300 tokens with overlap; good general-purpose default.
• Overlap – Repeats a portion of adjacent chunks to avoid cutting off meaning at boundaries.
• Semantic chunking – Splits by meaning (sentences/sections) instead of fixed size; improves retrieval quality for structured text.
• Hierarchical chunking
  • Embed small child chunks for precise matching
  • Return larger parent chunks at retrieval time for richer context
  • Reduces total tokens sent to the FM while preserving local context
• When documents are well-structured (headings/sections) → hierarchical or semantic chunking

Exam rule:

• Poor answers ≠ bad model → often bad chunking

Chunking vs vector store behavior

• Vector stores index vectors, not raw text.
• Retrieval returns associated text + metadata, not just the vector span.
• Metadata filtering reduces the candidate set before ANN search, improving relevance and performance.

Metadata in Bedrock Knowledge Bases

• metadata.json – Optional file that accompanies documents in a Knowledge Base.
• Used to attach structured attributes (tenant, product, region, doc type, ACL hints) to each chunk.
• Enables:
  • Metadata-based filtering during retrieval
  • Access control at retrieval time (not IAM)
  • Explainability / traceability (why this chunk was returned)
• Metadata is stored with embeddings and travels through retrieval.

Exam gotchas:

• Metadata is optional, but required for filtering and multi-tenant RAG.
• Metadata filtering ≠ Guardrails and ≠ IAM.
• IAM controls access to the KB; metadata controls what gets retrieved.

Chunking & RAG rules of thumb

• Large documents, generic answers → increase chunk size
• Precise questions, factual lookup → smaller chunks + overlap
• Need surrounding context → hierarchical chunking
• Multi-tenant or scoped retrieval → metadata.json
• Hallucinations with “correct” retrieval → chunking strategy issue, not model choice

Orchestration & workflows

• AWS Step Functions – Orchestrates stateful workflows. Often used to chain data ingestion, embedding, calling FMs, and storing outputs.
• Lambda – Event‑driven compute; used for chunking text, generating embeddings or gluing services together.
• API Gateway – Exposes a REST/HTTP interface for your GenAI application.
• EventBridge – Bus for event‑driven architectures.
• AppConfig – For runtime feature flags and dynamic model selection; can be used to switch FMs based on criteria.

Security & governance patterns

• Threats: Prompt injection, data exfiltration, tool misuse. Always sanitise user inputs, restrict tool access and implement guardrails.
• Least privilege: Use fine‑grained IAM policies, role assumption and scoped credentials. For multi‑tenant systems, isolate per‑tenant data sources and encryption keys.
• Encryption: Use KMS for data at rest; enforce TLS in transit; store embeddings in encrypted buckets or databases.
• Network isolation: Use VPC endpoints/PrivateLink to call Bedrock or SageMaker privately; configure security groups and subnets.
• Auditability: Log prompts, responses and tool invocations via CloudWatch and AWS CloudTrail.
• Guardrails & A2I: For high‑risk tasks, implement content filters and send outputs for human review.

System Resiliency Patterns (GenAI workloads)

• Chain-of-Thought instructions
  • Encourage structured reasoning for complex tasks
  • Improves accuracy and consistency (use carefully; avoid exposing reasoning verbatim)
• Retry & failure handling
  • Exponential Backoff for transient model or service failures
  • Circuit Breaker pattern to prevent cascading failures
    • Common implementation: Step Functions + DynamoDB
• Goal: graceful degradation, not hard failure, when models or downstream services misbehave

Humans in the Loop (HITL) & Quality Control

• Human Augmentation → AI drafts, humans refine (review/edit before final output).
• Escalation Criteria → Route uncertain cases (e.g., low confidence scores) to human experts.
• User feedback loop
  • Collect via API Gateway
  • Store/index in DynamoDB
  • Use to measure model/variant preference and drive continuous improvement
• Common use cases:
  • Regulated decisions
  • Ambiguous classifications
  • High-impact outputs where correctness > latency

Designing RAG pipelines

1. Ingest & chunk: Use Glue, Lambda, or custom scripts to extract data from documents, chunk text (size/overlap matters for recall), and pre‑process.
2. Generate embeddings: Use Bedrock embedding APIs or frameworks like SentenceTransformers; decide on vector dimension.
3. Store embeddings: Choose a vector store (OpenSearch, S3 Vectors, Aurora, Pinecone, etc.) based on dataset size and latency requirements.
4. Retrieve relevant chunks: Perform vector search (may combine with keyword search for hybrid retrieval).
5. Ground responses: Provide retrieved context to the FM with instructions to use only that information; enforce via guardrails and grounding checks.
6. Evaluate & refine: Use evaluation datasets, human feedback and metrics (BERTScore, ROUGE) to iterate and catch hallucinations.

Multi‑tenant GenAI considerations

• Tenant isolation: Separate data sources, embeddings and encryption keys per tenant; filter queries by tenant ID.
• Per‑tenant access control: Enforce IAM and RBAC at retrieval and tool layers.
• No cross‑tenant training: Do not mix tenant data in fine‑tuning unless explicit permission.
• Observability: Monitor usage and errors by tenant; alert on anomalies.

General Tips

• Change FM model without code changes → AWS AppConfig or Bedrock Intelligent Prompt Routing / Router Agent
• Evaluate RAG quality end-to-end → Bedrock Model Evaluations with retrieve-and-generate evaluation jobs
• Score correctness, completeness, faithfulness, coherence → Evaluator model (LLM-as-a-judge)
• Well-defined steps, branching logic, auditable execution → AWS Step Functions
• Track trained model versions and approvals → Amazon SageMaker Model Registry
• Track prompt templates with versioning, approval workflows → Amazon Bedrock Prompt Management
• Auditable history of API access → AWS CloudTrail
• Show data source origin, schema, lineage → AWS Glue Data Catalog
• One-off or exploratory data cleanup (UI-driven) → SageMaker Data Wrangler
• Automated or recurring data cleanup → AWS Glue ETL
• Detect and monitor bias or explain predictions (training data / not prompt) → Amazon SageMaker Clarify
• Enforce model version governance and documentation → SageMaker model governance (Model Registry + model cards)
• Knowledge Base ingestion troubleshooting → CloudWatch Logs + Logs Insights

Bedrock Model Evaluation

1. Define evaluation metrics → correctness, completeness, faithfulness, fluency
2. Prepare evaluation dataset (S3) → prompts + reference answers (and reference contexts for RAG)
3. Run Bedrock Model Evaluation jobs → use LLM-as-a-judge (evaluator model) to score outputs automatically
4. Apply quality gates → thresholds + approval workflow via AWS Step Functions
5. Finalize decision with evaluation report → compare models (baseline vs candidate) and approve promotion

Key rules of thumb

• Automated scoring → Bedrock Model Evaluations (not CloudWatch, not manual review)
• Correctness / faithfulness metrics → Evaluator FM (LLM-as-judge)
• Human approval required → Step Functions gate (not just dashboards)
• RAG evaluation → use retrieve-and-generate jobs, not retrieve-only

Bedrock Guardrails Observability

• Detect interventions → InvocationIntervened metric
• Identify input vs output trigger → GuardrailContentSource
• Identify exact policy fired → Guardrail tracing + GuardrailPolicyType
• Tune guardrails safely → Tracing required
• Explain customer-facing blocks → Tracing (not metrics alone)
• Test guardrails offline → Model Evaluation jobs

Data, Governance, and Auditability

• Custom domain rule checking → AWS Lambda

• Auditable access → CloudTrail + IAM (not custom application logs)

• Tracking S3 data sources and lineage → AWS Glue Data Catalog

• Regulated industries → Glue Data Catalog, CloudTrail, metadata tags, IAM-based access control

• Data cleaning, PII masking, intent classification before LLMs → AWS Lambda + Amazon Comprehend (not Guardrails, not Macie)

  • Exam gotcha: On most AWS exams, PII → Macie. In Bedrock / GenAI flows, pre-model PII → Comprehend.

• Blocking malformed, abusive, or obviously malicious requests → Amazon API Gateway

  • Use when the question mentions “before backend services”, “request validation”, or “first line of defense”
  • API Gateway handles structure & pattern enforcement, not semantic understanding
  • Never replaces Comprehend or Guardrails

• Rule:
  At the edge / before compute → API Gateway (schema, size, regex, allow/deny)
  Before the model → Lambda + Comprehend (PII + intent)
  At invocation → Guardrails (LLM behavior & output)
  At rest → Macie

Networking and Security

• Secure private service access → VPC endpoints / PrivateLink
• On-prem execution → AWS Outposts
• 5G / edge workloads → AWS Wavelength

RAG Quality, Explainability, and Caching

• RAG explainability → propagate metadata into embeddings
• Reduce hallucinations → RetrieveAndGenerate with Bedrock Knowledge Bases
• Retrieve-only RAG evaluation → measure retrieval quality independent of generation
• Hierarchical chunking → small child chunks for search, return larger parent chunks for context
• Use hierarchical chunking when documents are sectioned and answers need surrounding context

Performance and Cost Optimization

• Massive datasets + throttling + idle compute → use Bedrock Batch Inference (not InvokeModel)
• Static or repeated prompt content → Bedrock prompt caching
• Identical public requests → CloudFront edge cache
• Similar but not identical requests → semantic cache

Streaming and Real-Time Use Cases

• Real-time token streaming + serverless → API Gateway WebSocket + Lambda
• High-volume real-time ingestion → Kinesis Data Streams
• Near-real-time delivery → Kinesis Firehose

Agents and Tooling

• Agents should not speak REST
• If agents call strict or mutable external APIs → place an MCP tool boundary in front
• Validate arguments before calling the external API
• MCP standardizes interfaces, not compute → deploy each MCP server on compute that matches workload

Data Movement

• Large data transfers (on-prem ↔ AWS or AWS ↔ AWS) → AWS DataSync