How to Build RAG With Amazon Bedrock Knowledge Bases (2026 Guide)

This guide is for engineers who want a working retrieval-augmented generation (RAG) setup on AWS without hand-building an embedding pipeline. By the end you will have documents in an S3 bucket, a Bedrock knowledge base that chunks and embeds them with Titan Text Embeddings V2 into an Amazon S3 Vectors index, a scripted sync, and a Python function that answers questions with citations through the RetrieveAndGenerate API using Claude Sonnet 5.5. Every request in this post was checked against the service models shipped in boto3 1.43.127 (the latest on PyPI on 10 November 2026) using botocore's own parameter validator; I did not run it against a live account for this write-up, so expect to adjust Region and model availability for your account. You need an AWS account where you can create IAM roles, Python 3.10 or newer, and permission to use Bedrock in a Region such as us-east-1. If you are still deciding between clouds, read Bedrock vs Azure Foundry vs Vertex AI (2026) first.

Step 1: Decide between a managed and a customer-managed knowledge base

AWS now offers two kinds of knowledge base, and the docs recommend the first for most people. A Managed Knowledge Base hides the vector store entirely: Bedrock handles parsing, embedding, reranking and storage, adds connectors for SharePoint, Confluence, Google Drive and OneDrive, and enforces document-level permissions at retrieval time. The pricing page lists it at $5.00 per GB of raw data per month for index storage and $1.00 per 1,000 standard retrieval calls, with parsing, embeddings and reranking included.

A customer-managed (vector store) knowledge base is what this guide builds. You pick the embedding model and the vector store, you control chunking, and you pay the vector store's own bill. Choose it when you need to know exactly where vectors live, want to share the index with other code, or need the cheapest storage for a large corpus. The API calls are nearly identical, so moving later is not painful.

Step 2: Choose a vector store and an embedding model

The vector store prerequisites page lists the supported stores. These three cover most AWS shops:

Vector store	Minimum cost to keep it running	Strengths	Watch out for	Source
Amazon S3 Vectors	No floor: $0.06 per GB-month, $0.20 per GB uploaded, $2.50 per million queries plus a small per-TB data-processed fee (us-east-1)	Cheapest at rest, no capacity planning	Float32 only, 1 KB custom metadata and 35 keys per vector, no hybrid text search	S3 pricing
OpenSearch Serverless	Classic collections bill at least 2 OCUs at $0.24 per OCU-hour, about $350 a month; NextGen collections scale to zero after 10 idle minutes	Hybrid search, binary vectors, quick-create from the console	Vector collections cannot share OCUs with other collections	OpenSearch pricing
Aurora PostgreSQL with pgvector	Your Aurora cluster's cost	SQL access to the same rows, metadata filters, GIN text index for hybrid search	Must be in the same account; schema fields cannot change after creation	Bedrock docs

I use S3 Vectors below because it has no idle cost and needs nothing more than two API calls to set up. If you already run Postgres and want to own the SQL, the pgvector RAG guide covers that route, and Vector Databases in 2026 explains when a dedicated store earns its keep.

For embeddings, the supported models page lists Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0, 256, 512 or 1,024 dimensions), Cohere Embed English and Multilingual v3 (1,024 dimensions) and Amazon Nova Multimodal Embeddings for images, audio and video. Titan V2 at 1,024 dimensions is a sensible default for English and mixed technical text. The dimension you pick is permanent for the index, so decide now.

Install the pinned SDK in a virtual environment:

python3 -m venv .venv && source .venv/bin/activate
pip install "boto3==1.43.127"
python -c "import boto3; print(boto3.__version__)"
# 1.43.127

Step 3: Upload documents and metadata to S3

The bucket must be a general purpose S3 bucket in the same Region as the knowledge base, according to the S3 connector docs. Put documents under a prefix so you can scope the data source to it, and add a sidecar metadata file named after each document with .metadata.json appended. Metadata files must stay under 10 KB.

export AWS_REGION=us-east-1
export DOCS_BUCKET=my-kb-docs-123456789012   # placeholder: use your own bucket name

aws s3 mb "s3://${DOCS_BUCKET}" --region "${AWS_REGION}"
aws s3 cp ./runbooks/ "s3://${DOCS_BUCKET}/docs/runbooks/" --recursive

# Sidecar metadata for one file: docs/runbooks/cloudtrail-retention.md
cat > cloudtrail-retention.md.metadata.json <<'EOF'
{
  "metadataAttributes": {
    "team": "platform",
    "doc_type": "runbook",
    "reviewed_year": 2026
  }
}
EOF
aws s3 cp cloudtrail-retention.md.metadata.json \
  "s3://${DOCS_BUCKET}/docs/runbooks/cloudtrail-retention.md.metadata.json"

This simplified format stores the attributes for filtering without mixing them into the embedding. If you want an attribute to influence similarity, use the long form with "includeForEmbedding": true, shown in the same AWS page.

Step 4: Create the S3 vector bucket and index

The index needs the same dimension as your embedding model, and Bedrock needs two non-filterable metadata keys, AMAZON_BEDROCK_TEXT and AMAZON_BEDROCK_METADATA, or long chunks will blow through the per-vector metadata limit.

# create_vector_index.py  (boto3 1.43.127)
import boto3

REGION = "us-east-1"
VECTOR_BUCKET = "kb-demo-vectors"
INDEX_NAME = "docs-titan-v2-1024"

s3v = boto3.client("s3vectors", region_name=REGION)

s3v.create_vector_bucket(
    vectorBucketName=VECTOR_BUCKET,
    encryptionConfiguration={"sseType": "AES256"},  # or "aws:kms" plus kmsKeyArn; cannot change later
)

s3v.create_index(
    vectorBucketName=VECTOR_BUCKET,
    indexName=INDEX_NAME,
    dataType="float32",          # S3 Vectors supports float32 only
    dimension=1024,              # must match Titan Text Embeddings V2 setting below
    distanceMetric="cosine",
    metadataConfiguration={
        "nonFilterableMetadataKeys": ["AMAZON_BEDROCK_TEXT", "AMAZON_BEDROCK_METADATA"]
    },
)
print(s3v.get_index(vectorBucketName=VECTOR_BUCKET, indexName=INDEX_NAME)["index"]["indexArn"])
# arn:aws:s3vectors:us-east-1:123456789012:bucket/kb-demo-vectors/index/docs-titan-v2-1024

Step 5: Create the knowledge base service role

Bedrock assumes this role to read S3, call the embedding model and write vectors. The policies below follow the service role page, scoped to one bucket prefix and one index. Replace the account ID.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "bedrock.amazonaws.com" },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": { "aws:SourceAccount": "123456789012" },
        "ArnLike": { "AWS:SourceArn": "arn:aws:bedrock:us-east-1:123456789012:knowledge-base/*" }
      }
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EmbedWithTitanV2",
      "Effect": "Allow",
      "Action": "bedrock:InvokeModel",
      "Resource": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"
    },
    {
      "Sid": "ListDocsBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::my-kb-docs-123456789012",
      "Condition": { "StringEquals": { "aws:ResourceAccount": "123456789012" } }
    },
    {
      "Sid": "ReadDocsPrefix",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::my-kb-docs-123456789012/docs/*",
      "Condition": { "StringEquals": { "aws:ResourceAccount": "123456789012" } }
    },
    {
      "Sid": "WriteVectors",
      "Effect": "Allow",
      "Action": [
        "s3vectors:PutVectors",
        "s3vectors:GetVectors",
        "s3vectors:DeleteVectors",
        "s3vectors:QueryVectors",
        "s3vectors:GetIndex"
      ],
      "Resource": "arn:aws:s3vectors:us-east-1:123456789012:bucket/kb-demo-vectors/index/docs-titan-v2-1024"
    }
  ]
}

aws iam create-role --role-name BedrockKbDocsRole \
  --assume-role-policy-document file://kb-trust.json
aws iam put-role-policy --role-name BedrockKbDocsRole \
  --policy-name kb-docs-access --policy-document file://kb-permissions.json

One sentence in the AWS docs deserves a highlighter: data source and ingestion calls do not pass a role, so anyone who can manage data sources can ingest whatever this role can read and then query it. Keep the role's S3 access to exactly the prefixes you mean to index. The IAM explainer covers why trust policies and condition keys matter here.

Step 6: Create the knowledge base and the S3 data source

# create_kb.py  (boto3 1.43.127)
import time
import boto3

REGION, ACCOUNT = "us-east-1", "123456789012"
INDEX_ARN = f"arn:aws:s3vectors:{REGION}:{ACCOUNT}:bucket/kb-demo-vectors/index/docs-titan-v2-1024"

agent = boto3.client("bedrock-agent", region_name=REGION)

kb = agent.create_knowledge_base(
    name="docs-kb",
    roleArn=f"arn:aws:iam::{ACCOUNT}:role/BedrockKbDocsRole",
    knowledgeBaseConfiguration={
        "type": "VECTOR",
        "vectorKnowledgeBaseConfiguration": {
            "embeddingModelArn": f"arn:aws:bedrock:{REGION}::foundation-model/amazon.titan-embed-text-v2:0",
            "embeddingModelConfiguration": {
                "bedrockEmbeddingModelConfiguration": {"dimensions": 1024, "embeddingDataType": "FLOAT32"}
            },
        },
    },
    storageConfiguration={
        "type": "S3_VECTORS",
        "s3VectorsConfiguration": {"indexArn": INDEX_ARN},  # indexArn alone, or vectorBucketArn plus indexName
    },
)["knowledgeBase"]
kb_id = kb["knowledgeBaseId"]

while agent.get_knowledge_base(knowledgeBaseId=kb_id)["knowledgeBase"]["status"] == "CREATING":
    time.sleep(5)

ds = agent.create_data_source(
    knowledgeBaseId=kb_id,
    name="docs-s3",
    dataDeletionPolicy="DELETE",  # remove vectors when the data source is deleted
    dataSourceConfiguration={
        "type": "S3",
        "s3Configuration": {
            "bucketArn": f"arn:aws:s3:::my-kb-docs-{ACCOUNT}",
            "bucketOwnerAccountId": ACCOUNT,
            "inclusionPrefixes": ["docs/"],
        },
    },
    vectorIngestionConfiguration={
        "chunkingConfiguration": {
            "chunkingStrategy": "FIXED_SIZE",
            "fixedSizeChunkingConfiguration": {"maxTokens": 300, "overlapPercentage": 15},
        }
    },
)["dataSource"]
print("KB_ID=%s DS_ID=%s" % (kb_id, ds["dataSourceId"]))

The chunking page says default chunking produces roughly 300-token chunks on sentence boundaries, so the fixed-size setting above mostly makes that explicit and adds overlap. You cannot change the chunking strategy after the data source exists, which is why it is pinned in code. AWS also advises against hierarchical chunking with S3 Vectors, because parent and child context is stored as metadata and can hit the size limit.

Step 7: Sync the data source and check the job

Every sync is an ingestion job. After the first full crawl, the S3 connector processes only new, modified and deleted objects.

# sync.py  (boto3 1.43.127)
import sys, time, uuid
import boto3

agent = boto3.client("bedrock-agent", region_name="us-east-1")
kb_id, ds_id = sys.argv[1], sys.argv[2]

job = agent.start_ingestion_job(
    knowledgeBaseId=kb_id, dataSourceId=ds_id, clientToken=str(uuid.uuid4())
)["ingestionJob"]

while True:
    j = agent.get_ingestion_job(
        knowledgeBaseId=kb_id, dataSourceId=ds_id, ingestionJobId=job["ingestionJobId"]
    )["ingestionJob"]
    if j["status"] in ("COMPLETE", "FAILED", "STOPPED"):
        break
    time.sleep(10)

print(j["status"], j["statistics"])
for reason in j.get("failureReasons", []):
    print("FAIL:", reason)
sys.exit(0 if j["status"] == "COMPLETE" and j["statistics"]["numberOfDocumentsFailed"] == 0 else 1)

A healthy first run prints something shaped like COMPLETE {'numberOfDocumentsScanned': 42, 'numberOfNewDocumentsIndexed': 42, 'numberOfDocumentsFailed': 0, ...}. Check numberOfDocumentsFailed as well as the status, and treat anything above zero as a failed build; the script exits non-zero in that case. Run it from CI or an EventBridge schedule after uploads, and avoid starting overlapping jobs on the same data source.

Step 8: Ask questions with RetrieveAndGenerate

RetrieveAndGenerate runs retrieval and generation in one call and returns citations. For the model, pass an inference profile ARN: the Claude Sonnet 5.5 model card lists knowledge base support but no in-Region option in us-east-1, only the us. geographic and global. profiles. I use us. here to keep processing inside the US; the cross-Region inference page explains the trade-off.

# ask.py  (boto3 1.43.127)
import sys
import boto3

REGION, ACCOUNT = "us-east-1", "123456789012"
MODEL_ARN = f"arn:aws:bedrock:{REGION}:{ACCOUNT}:inference-profile/us.anthropic.claude-sonnet-5-5"

rt = boto3.client("bedrock-agent-runtime", region_name=REGION)

def ask(kb_id: str, question: str, team: str | None = None) -> None:
    vector_cfg = {"numberOfResults": 6}
    if team:
        vector_cfg["filter"] = {"equals": {"key": "team", "value": team}}

    resp = rt.retrieve_and_generate(
        input={"text": question},
        retrieveAndGenerateConfiguration={
            "type": "KNOWLEDGE_BASE",
            "knowledgeBaseConfiguration": {
                "knowledgeBaseId": kb_id,
                "modelArn": MODEL_ARN,
                "retrievalConfiguration": {"vectorSearchConfiguration": vector_cfg},
                "generationConfiguration": {
                    "inferenceConfig": {"textInferenceConfig": {"maxTokens": 800, "temperature": 0.0}}
                },
            },
        },
    )
    print(resp["output"]["text"], "\n")
    for c in resp.get("citations", []):
        for ref in c["retrievedReferences"]:
            print("  source:", ref["location"].get("s3Location", {}).get("uri"))

if __name__ == "__main__":
    ask(sys.argv[1], sys.argv[2], team="platform")

python ask.py KB12345678 "How long do we keep CloudTrail logs, and where?"

When an answer looks wrong, call Retrieve on its own and read the chunks before you touch the prompt. In my experience the problem is usually retrieval, not generation, which is the theme of Why RAG Fails in Production.

hits = rt.retrieve(
    knowledgeBaseId="KB12345678",
    retrievalQuery={"text": "cloudtrail retention"},
    retrievalConfiguration={"vectorSearchConfiguration": {"numberOfResults": 5}},
)["retrievalResults"]
for h in hits:
    print(round(h.get("score", 0), 3), h["location"]["s3Location"]["uri"], h["content"]["text"][:120])

Your application's role (not the knowledge base role) needs bedrock:RetrieveAndGenerate, bedrock:Retrieve on the knowledge base ARN, and bedrock:InvokeModel on both the inference profile and the underlying foundation model in each destination Region of the profile. AWS's S3 connector page warns that all synced data is available to anyone with bedrock:Retrieve on the knowledge base, so one knowledge base per audience is simpler than trying to filter your way to access control.

Step 9: Estimate the bill before you scale

For a customer-managed knowledge base on S3 Vectors you pay for four things: embedding tokens during each sync (Titan V2 on-demand rates on the Bedrock pricing page), S3 Vectors storage, PUTs and queries at the rates in the table above, the generation model's input and output tokens on every question, and any Guardrails you attach ($0.15 per 1,000 text units for content filters). Retrieved chunks are input tokens, so six 300-token chunks add roughly 1,800 tokens to every question before your prompt even starts. The cheapest lever is numberOfResults; the second is caching your system prompt, covered in How to Cut LLM API Costs.

If you picked OpenSearch Serverless instead, the floor dominates small projects: two OCUs at $0.24 per hour is about $350 a month before a single query, unless you use the NextGen collection type that scales to zero.

Troubleshooting

Symptom	Likely cause	Fix
AccessDeniedException on CreateKnowledgeBase	Caller lacks iam:PassRole on the KB role, or trust policy SourceArn does not match	Grant iam:PassRole for BedrockKbDocsRole; check Region and account in the trust policy
Ingestion job FAILED with S3 access errors	KB role cannot list the bucket or read the prefix, or bucket is KMS-encrypted	Fix the s3:ListBucket and s3:GetObject statements; add kms:Decrypt with kms:ViaService for S3
ValidationException naming the index when creating the knowledge base	Mixed identifiers in s3VectorsConfiguration (for example indexArn together with indexName)	Pass indexArn alone, or vectorBucketArn plus indexName
Ingestion fails on metadata size with S3 Vectors	Long chunks or hierarchical chunking exceed per-vector metadata limit	Add AMAZON_BEDROCK_TEXT and AMAZON_BEDROCK_METADATA as non-filterable keys; use fixed-size chunks
Dimension mismatch errors at ingestion	Index dimension differs from the embedding model configuration	Recreate the index with the model's dimension (1,024 for Titan V2 here)
Job COMPLETE but numberOfDocumentsFailed above zero	Unsupported file type, file over the size quota, or bad metadata JSON	Read failureReasons; check Bedrock knowledge base quotas; validate each .metadata.json
Metadata filter returns nothing	Attribute missing from metadata file, wrong key name, or file not named <file>.metadata.json	Run Retrieve without the filter and inspect metadata on results; re-sync after fixing names
RetrieveAndGenerate access denied on the model	Missing bedrock:InvokeModel on the inference profile or a destination Region's foundation model, or an SCP blocks a destination Region	Allow both ARNs; allow all destination Regions listed for the profile
Answers ignore newly uploaded files	No sync ran after the upload	Run the Step 7 script after every upload batch, or schedule it

FAQ

What is the difference between Retrieve and RetrieveAndGenerate?

Retrieve returns the matching chunks, their scores and source locations, and leaves generation to you. RetrieveAndGenerate does retrieval and calls a model in one request, returning an answer with citations. Use Retrieve for debugging and when you want full control over the prompt.

Which vector store is cheapest for a Bedrock knowledge base?

For small or bursty workloads, Amazon S3 Vectors, because it has no minimum: storage is $0.06 per GB-month in us-east-1 and queries are billed per call. OpenSearch Serverless classic collections bill at least two OCUs, roughly $350 a month, though its NextGen collections can scale to zero.

How do I keep a Bedrock knowledge base in sync with S3?

Call StartIngestionJob after each upload batch or on a schedule. After the first full crawl, the S3 connector only processes added, modified and deleted objects, so frequent syncs are cheap.

Can I change the embedding model or chunking later?

Not in place. The chunking strategy is fixed once the data source exists, and the vector dimension is fixed by the index. Create a new index and data source, sync, switch your application to the new knowledge base ID, then delete the old one.

What I check first in production is boring and always the same: the ingestion job statistics after every sync, the raw Retrieve output for a fixed set of ten questions, and the CloudTrail additionalEventData.inferenceRegion field to confirm the model ran where we said it would. Wire those three into CI and most RAG regressions show up before a user sees them. The official references are the Bedrock Knowledge Bases user guide, the vector store setup page and the service role permissions page.

Related: How to Build a RAG Pipeline With Postgres and pgvector (2026), Vector Databases in 2026: Do You Need One, or Is pgvector Enough? and How to Cut LLM API Costs: Prompt Caching, Batching, Routing (2026).