$ show aws-samples/bedrock-access-gateway
This repository implements an OpenAI-compatible REST API proxy for Amazon Bedrock, enabling applications built for OpenAI SDKs to seamlessly use Bedrock foundation models without code changes. The FastAPI-based service translates standard OpenAI API calls (chat completions, embeddings, models listing) into Amazon Bedrock runtime invocations and returns responses in OpenAI-compatible format. The architecture supports two deployment options: Lambda with API Gateway (serverless, cost-effective) or Fargate with ALB (lowest latency, no cold starts). Key features include streaming responses via SSE, tool/function calling, multimodal inputs, prompt caching for Claude/Nova models, reasoning mode for Claude 3.7 and DeepSeek R1, and interleaved thinking for Claude 4 models. Infrastructure is managed via CloudFormation templates with proper IAM roles and Secrets Manager integration for API key authentication. The codebase demonstrates solid engineering practices with clean separation of concerns, proper error handling, and comprehensive feature support. However, the absence of test files and limited documentation reduce overall quality. The project is actively maintained with recent commits and up-to-date dependencies.
The LLM identified 5 files worth highlighting for reuse:
Core Bedrock model implementation demonstrating excellent craftsmanship. The BedrockModel class implements feature detection for prompt caching, reasoning mode, and interleaved thinking through pattern matching rather than hardcoded whitelists. Handles profile resolution for cross-region and application inference profiles without regex. The _parse_messages method correctly reformats OpenAI message structures to Bedrock Converse API format, including handling of tool calls, multimodal content, and assistant message prefill scenarios. Proper async/await patterns with thread pool execution for blocking boto3 calls.
Authentication layer supporting multiple API key sources with graceful fallback: AWS Secrets Manager (preferred), SSM Parameter Store (backward compatibility), and environment variables. Implements HTTP Bearer token validation with proper error handling. The pattern of trying multiple sources with clear error messages is reusable for any service requiring flexible credential management.
Comprehensive Pydantic models defining the OpenAI-compatible API contract. Includes proper type hints, field validation, and support for streaming responses with usage tracking. The models handle edge cases like reasoning_content for reasoning mode, tool_calls for function calling, and prompt_tokens_details for cache metrics. Well-structured for reuse in any OpenAI-compatible proxy service.
CloudFormation template for Lambda deployment with API Gateway response streaming. Demonstrates proper IAM least-privilege principles with specific Bedrock permissions for foundation models, inference profiles, and application inference profiles. The template includes Lambda Web Adapter configuration for streaming support and proper timeout settings for long-running model responses. Reusable pattern for serverless API deployments.
Build and push script with proper error handling (set -o errexit, nounset, pipefail). Includes important Docker build flags for Lambda compatibility (--provenance=false, --sbom=false) to avoid OCI manifest issues. The script handles ECR repository creation, authentication, and image tagging with user confirmation prompts. Reusable pattern for AWS container deployments.