Universal Multi-Provider AI Gateway & AWS Bedrock SigV4 Bridge (BYOK)
Route Claude 3.5, Gemini 2.0, Llama 3, and GPT-4o with standard OpenAI SDKs or Bring-Your-Own-Key (BYOK). Translate Bearer tokens to Amazon Bedrock AWS SigV4 in volatile memory, stream completions with sub-millisecond overhead, and slash token costs by up to 88% with Aurora pgvector semantic caching.
curl -X POST https://gateway.kiyaslabs.tech/v1/chat/completions \
-H "Authorization: Bearer kl_beta_public_sandbox" \
-H "Content-Type: application/json" \
-d '{"model": "anthropic.claude-3-5-sonnet", "messages": [{"role": "user", "content": "Ping"}]}'18ms Vector Cache vs 1,480ms Cold Inference
98.8% latency reduction vs. raw Bedrock Claude 3.5 Sonnet inference.
Vector similarity match served directly from Aurora Serverless.
Detailed Component Pipeline Timing
Calculate Your Monthly LLM Infrastructure Savings
Without semantic cache offloading
58.0M tokens served free from cache
$5,220 annualized cost reduction
Eliminated client wait-time for end users every single month.
Amazon Titan Embeddings v2 with HNSW cosine distance > 0.85.
Built for High-Throughput Production & Zero Prompt Retention
Vector Semantic Caching
Analyzes semantic intent rather than brittle exact-string hashing. Computes high-dimensional vector representations using Amazon Titan Embeddings v2 and queries Amazon Aurora Serverless with pgvector HNSW indexing in under 2ms.
- HNSW Cosine similarity index
- Sub-20ms P99 cache lookup
- Tenant-isolated vector partitions
- Automated TTL & eviction policies
AWS SigV4 Bedrock Bridge
Acts as an autonomous translation bridge between standard OpenAI/Anthropic SDK headers (Bearer sk-...) and Amazon Bedrock’s mandatory AWS Signature Version 4 protocol. Signs requests in volatile memory using HMAC-SHA256.
- Drop-in base_url replacement
- In-memory SigV4 signing (< 0.5ms overhead)
- Zero AWS SDK dependencies on client
- Automatic regional IAM role assumption
Real-Time Token FinOps
Enforces granular per-tenant token quotas, sliding-window rate limits, and circuit breakers against infinite loop prompts. Delivers sub-second telemetry aggregation and cost attribution across teams.
- Hard & soft token budget ceilings
- Automatic runaway prompt throttling
- Model fallover routing on 429/503
- Live cost attribution by team/project
PII Scrubbing & Zero-Retention
Zero-Prompt Persistence architecture: raw prompts and completions exist exclusively in volatile RAM buffers during active streaming and are cryptographically zeroized upon stream closure. Zero disk writes, zero model training.
- Regex & NER PII masking in-flight
- Zero prompt storage on persistent disk
- AWS KMS envelope key encryption
- Strict HIPAA BAA & SOC2 Type II alignment
Drop-in Compatibility with OpenAI SDK, LangChain & cURL
# Install: pip install openai
from openai import OpenAI
# Simply change the base_url to PromptRelay's gateway
client = OpenAI(
base_url="https://gateway.kiyaslabs.tech/v1",
api_key="kl_live_bedrock_demo" # Your PromptRelay Ephemeral Key
)
# Call Bedrock Claude 3.5 Sonnet using standard OpenAI syntax
completion = client.chat.completions.create(
model="anthropic.claude-3-5-sonnet",
messages=[{"role": "user", "content": "Analyze AWS SigV4 bridge benefits"}],
stream=True,
extra_headers={
"x-promptrelay-cache-threshold": "0.85",
"x-promptrelay-fallback-models": "anthropic.claude-3-haiku,meta.llama3-70b"
}
)
for chunk in completion:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)https://gateway.kiyaslabs.tech/v1Compatible with Python 3.9+, Node.js 18+, Bun, DenoEnterprise 3-Tier Infrastructure with KMS Envelope Encryption
CloudFront Anycast Edge
Global edge presence terminating TLS 1.3 with strict forward secrecy ciphers. Provides automated AWS Shield DDoS mitigation and intelligent regional routing to nearest compute.
- TLS 1.3 Strict Forward Secrecy
- Global Anycast Geo-Routing
- AWS Shield Standard DDoS Mitigation
- HTTP/2 & HTTP/3 multiplexing
ECS Fargate Relay Engine
Containerized relay workers operating entirely in volatile memory. Converts Bearer tokens to AWS SigV4 HMAC-SHA256 signatures and tees SSE streaming tokens to background workers.
- AWS SigV4 in-memory translation
- Asynchronous stream-teeing engine
- Volatile RAM ring buffers (zero disk)
- Per-tenant token FinOps rate limiting
Aurora pgvector & KMS Vault
Amazon Aurora Serverless PostgreSQL with pgvector HNSW indexing for high-speed semantic matching. Provider API keys are decrypted via AWS KMS envelope encryption exclusively in volatile RAM.
- Amazon Titan Embeddings v2
- Aurora pgvector HNSW cosine search
- AWS KMS Envelope Key Encryption
- Tenant-isolated row-level partitions
PromptRelay is currently in Public Beta (v0.4). Core routing and semantic caching are 100% free for early developer cohorts.
Transparent Pricing for Scaling AI Infrastructure
Developer Beta
ACTIVE NOWFull-featured access to the Universal Multi-Provider AI Gateway, AWS SigV4 bridge, and Aurora pgvector semantic cache.
- 10,000,000 tokens / month
- Sub-20ms semantic caching
- AWS SigV4 in-memory translation
- Zero-downtime Bedrock failover
- Standard community & Discord support
- Zero-Prompt Persistence guarantee
Production Team
POST-BETA PREVIEWDedicated rate limits, team token FinOps, and multi-region priority queues.
- 100,000,000 tokens / month
- Multi-tenant team cost attribution
- Custom cache similarity thresholds
- Priority low-latency relay queues
- Automated runaway prompt throttling
- Priority email & Slack support
Dedicated VPC
ENTERPRISESelf-hosted ECS Fargate proxy deployed directly inside your private AWS VPC.
- Unlimited token throughput
- Self-hosted AWS ECS proxy relay
- AWS VPC PrivateLink endpoints
- Custom HIPAA BAA & SOC2 addendum
- 99.99% availability SLA with financial credits
- Dedicated Solutions Architect support
Detailed Tier Comparison
Every tier guarantees Zero-Prompt Persistence and AWS SigV4 protocol support.
| Feature | Developer Beta ($0) | Production Team ($49) | Dedicated VPC (Custom) |
|---|---|---|---|
| Monthly Token Quota | 10M tokens | 100M tokens | Custom / Unlimited |
| AWS SigV4 Bedrock Bridge | Included | Included | Included |
| Aurora pgvector Semantic Cache | Shared Partition | Dedicated Partition | Customer VPC Aurora |
| Stream-Teeing Latency Overhead | < 1ms | < 0.5ms | < 0.2ms |
| Zero-Prompt Persistence | Volatile RAM only | Volatile RAM only | Volatile RAM only |
| AWS KMS Envelope Encryption | KMS Managed | Tenant-Scoped KMS | Customer-Owned CMK |
| Uptime SLA Commitment | Best Effort | 99.9% SLA | 99.99% Financial SLA |
| Support Channels | Discord & Community | Email & Shared Slack | 24/7 Dedicated AWS SA |
Frequently Asked Questions
Deploy PromptRelay into Your AWS VPC
Direct Technical Lead Inquiry
Dedicated Routing Channels
Enterprise Sandbox
enterprise@kiyaslabs.techProvisioning for enterprise subscriptions, dedicated VPC relays, and HIPAA BAAs.
Developer Support
support@kiyaslabs.techInquiries regarding the AWS SigV4 translation bridge, OpenAI SDK drop-in, or stream-teeing.
Security Disclosures
security@kiyaslabs.techEncrypted PGP channel for vulnerability reporting, pen-test reports, and compliance reviews.
Operating Entity for PromptRelay™ • Response SLA: Within 1 Business Day