Module Specification Writing Guide

# Azure OpenAI Module Specification ## Overview Azure OpenAI provides enterprise access to GPT-4, GPT-3.5-turbo, and embedding models. Key use cases: GenAI apps, RAG systems, code generation, semantic search. ## Architecture ``` User Application -> API Management (optional gateway) -> Azure OpenAI (Cognitive Services) |-- Deployments (gpt-4, embeddings) |-- Model versions \-- Rate limits per deployment ``` ## Key Capabilities - Multiple model deployments in single resource - Quota management (tokens/min, requests/min) - Virtual network integration - Customer-managed keys (CMK) - Content filtering and abuse monitoring ## Resource Inventory - **Cognitive Service Account** (Type: Microsoft.CognitiveServices/accounts) - OpenAI account - **Model Deployment** (Type: N/A) - Created in portal/Bicep - **Private Endpoint** (Type: Private Link) - Network isolation ## Configuration - **SKU**: Standard (pay-as-you-go) or Provisioned (reserved) - **Deployments**: Min 1, typical 3-5 (gpt-4, gpt-35-turbo, embedding-ada) - **Quota**: Set per deployment (50K-300K tokens/min recommended) - **Version Pinning**: Pin to specific model version (e.g., 0125-preview) ## Cost Estimation - **GPT-4 (Vision)**: $0.01-0.03 per 1K input tokens, $0.03-0.06 per 1K output - **GPT-3.5-turbo**: $0.0005-0.0015 per 1K input tokens - **Ada Embeddings**: $0.0001 per 1K tokens - **Estimate**: $2,000-5,000/month for 1M daily tokens ## SLAs & Limits - Availability: 99.9% SLA - Rate limits: 90K requests/min (Standard quota) - Model availability: Check regional availability - Context window: 4K-128K tokens (model-dependent) ## Bicep Template ```bicep resource cognitiveAccount 'Microsoft.CognitiveServices/accounts@2023-10-01-preview' = { name: 'openai-${uniqueSuffix}' location: location kind: 'OpenAI' sku: { name: 'S0' } properties: { customSubdomainName: 'openai-${uniqueSuffix}' apiProperties: { statisticsEnabled: true } publicNetworkAccess: 'Disabled' } } resource gpt4Deployment 'Microsoft.CognitiveServices/accounts/deployments@2023-10-01-preview' = { parent: cognitiveAccount name: 'gpt-4' properties: { model: { format: 'OpenAI', name: 'gpt-4', version: '0125-preview' } scaleSettings: { scaleType: 'Standard', capacity: 10 } } } ``` ## Well-Architected Framework - **Reliability**: Multi-region strategy, quota monitoring, retry logic - **Security**: Private endpoints, managed identities, input validation - **Cost**: Right-size quotas, monitor token usage, use cheaper models where possible - **Performance**: Batch API for non-realtime, caching embeddings, prompt optimization - **Operational Excellence**: Log all requests, set up alerts on quota exhaustion ## Best Practices 1. **Never hardcode API keys** - use Azure AD with managed identities 2. **Implement batching** - use Batch API for cost savings (50% cheaper) 3. **Cache embeddings** - reuse vector embeddings to reduce token cost 4. **Monitor token usage** - set up alerts when approaching quota limits 5. **Version models explicitly** - pin to specific version, test updates separately 6. **Implement retry logic** - handle rate limiting gracefully with exponential backoff 7. **Use embeddings cache** - store embeddings in cache (Redis/Cosmos) to avoid recomputation

# AI Search Module Specification ## Overview Azure AI Search (formerly Cognitive Search) provides full-text, vector, and hybrid search with semantic ranking. Typical uses include RAG backends, catalog search, knowledge bases, and enterprise document discovery. ## Architecture ``` Data Source (Blob, SQL, etc.) -> Indexer with Enrichment Skills -> Search Index (Vector + BM25) -> Semantic Reranker -> Search Results ``` ## Key Capabilities - Vector search with HNSW (Hierarchical Navigable Small World) - Hybrid search (full-text + vectors in single query) - Semantic ranking and re-ranking - Integrated vectorizers (Text-Embedding-3-small/large) - 100+ built-in enrichment skills - Security: Private endpoints, RBAC, customer-managed keys ## Resource Inventory | Resource | Type | Notes | |----------|------|-------| | Search Service | Microsoft.Search/searchServices | Main service | | Index | Search Index | Stores documents & vectors | | Indexer | Indexer | Data ingestion pipeline | | Data Source | Data Source | Points to Blob, SQL, Cosmos DB, etc | | Skill Set | Skill Set | OCR, entity extraction, etc | ## Configuration - **SKU**: Choose based on throughput and scale needs - **Partitions**: 1-12 (each adds ~25GB storage) - **Replicas**: 1-12 (scale for query throughput) - **Vector Config**: embedding field (Collection(Edm.Single), searchable) - **Semantic Config**: custom fields for re-ranking ## Cost Estimation - **Basic SKU**: $189/month + $12/partition - **S1 SKU**: $403/month + $12/partition - **S3 SKU**: $3,660/month + $12/partition (for massive indexes) - **Typical**: Basic + 2 partitions = ~$213/month for small RAG ## SLAs & Limits - Availability: 99.9% SLA (Standard SKUs) - Max documents per index: 2 billion - Max index size: Limited by partition count (~25GB per partition) - Max vector dimensions: 2048 - Query timeout: 30 seconds - Indexing throughput: Limited by SKU (Basic: ~10K docs/sec) ## Bicep Template ```bicep resource searchService 'Microsoft.Search/searchServices@2023-11-01' = { name: 'search-${uniqueSuffix}' location: location sku: { name: 'basic' } properties: { partitionCount: 2 replicaCount: 2 publicNetworkAccess: 'Disabled' networkRuleBypassOptions: 'AzureServices' authOptions: { aadOrApiKey: { aadAuthFailureMode: 'http401WithBearerChallenge' } } } } resource privateEndpoint 'Microsoft.Network/privateEndpoints@2023-11-01' = { name: 'pe-search-${uniqueSuffix}' location: location properties: { subnet: { id: subnetId } privateLinkServiceConnections: [ { name: 'search-connection' properties: { privateLinkServiceId: searchService.id groupIds: [ 'searchService' ] } } ] } } ``` ## Well-Architected Framework - **Reliability**: Multi-replica for HA, backup indexes, test failover - **Security**: Private endpoints, managed identities, data encryption - **Cost**: Right-size SKU (basic sufficient for RAG), use integrated vectorizer - **Performance**: Partition for scale, use vector search over full-text when possible - **Operational Excellence**: Monitor query latency, track indexing failures, alert on quota ## Best Practices 1. **Use hybrid search** - Combine full-text (BM25) + vector for accuracy 2. **Enable semantic ranking** - Cheap relative benefit for ranking quality 3. **Batch indexing** - Use batch API for bulk updates (not single-doc updates) 4. **Disable queryLanguage if not needed** - Reduces index size by 5-10% 5. **Use integrated vectorizer** - No need to manage embedding pipeline separately 6. **Monitor indexing lag** - Set up alerts when indexer falls behind 7. **Test vector dimensions** - 384D (Small) often sufficient; don't default to 3072D

# Azure AI Foundry Module Specification ## Overview Azure AI Foundry is a unified platform for building, evaluating, and deploying AI applications. It provides Prompt flow, Evaluations, and built-in safety governance. ## Key Components 1. **Hub** - Organization-level resource (single per org) 2. **Projects** - Team workspaces (multiple per hub) 3. **Prompt Flow** - Visual workflow for chaining models/APIs 4. **Evaluations** - Quality/safety metrics and benchmarking 5. **Content Filters** - Safety policies (hate, violence, sexual, self-harm) ## Resource Inventory - **AI Hub** (Type: Hub) - Organization-level - **AI Project** (Type: Project) - Team workspace - **Prompt Flow** (Type: Flow) - Orchestration - **Runtime** (Type: Compute) - Execution environment - **Content Filter** (Type: Policy) - Safety guardrails ## Configuration - **Hub Location**: Choose AIML-ready region - **Storage Account**: For artifacts and flows - **Key Vault**: For secrets management - **Hub Runtime**: For flow execution (or serverless) - **Safety Settings**: Enable content filters (default: all enabled) ## Cost Estimation - **Foundry Service**: $4/hour for compute (or pay-as-you-go) - **OpenAI Integration**: Billed separately per API calls - **Evaluations**: $0.01-0.10 per evaluation run - **Typical**: $200-500/month for active development team ## Bicep Template ```bicep resource aiHub 'Microsoft.MachineLearningServices/workspaces@2023-08-01' = { name: 'hub-${uniqueSuffix}' location: location kind: 'Hub' identity: { type: 'SystemAssigned' } properties: { description: 'AI Foundry Hub' storageAccount: storageId keyVault: kvId containerRegistry: acrId publicNetworkAccess: 'Disabled' } } resource aiProject 'Microsoft.MachineLearningServices/workspaces@2023-08-01' = { name: 'project-${uniqueSuffix}' location: location kind: 'Project' identity: { type: 'SystemAssigned' } properties: { description: 'AI Project under Hub' hubResourceId: aiHub.id storageAccount: storageId } } ``` ## Best Practices 1. **Organize by team** - One project per AI application/team 2. **Use Prompt Flow for orchestration** - Chain models, APIs, tools 3. **Evaluate before deployment** - Run evaluation suite on flow outputs 4. **Implement content filters** - Enable all safety filters in production 5. **Version control flows** - Commit flow YAML to Git 6. **Monitor safety metrics** - Track content filter triggers and adjust thresholds 7. **Document flow inputs/outputs** - Include examples in flow descriptions

# API Management Module Specification ## Overview API Management is an enterprise API gateway. For GenAI workloads: OpenAI gateway, token-aware rate limiting, request transformation, caching. ## Key Capabilities - Reverse proxy for OpenAI/AI Search/custom backends - Token-aware rate limiting policies - Request/response transformation (headers, body) - Caching (TTL-based for non-semantic requests) - API versioning and multiple backends - Developer portal and analytics ## Resource Inventory | Resource | Type | Purpose | |----------|------|---------| | APIM Instance | Microsoft.ApiManagement/service | API gateway | | Backends | Backend | OpenAI, AI Search, custom | | APIs | API | REST endpoints | | Policies | Policy | Transformation, rate limit | | Named Values | Secret | Connection strings, keys | ## Configuration Examples **For OpenAI Gateway:** - Backend: https://{{ openai-instance }}.openai.azure.com - Rate limit policy: X tokens/minute (not requests/minute) - Caching: Disable for LLM calls (non-deterministic) - Routing: /v1/chat/completions -> Azure OpenAI deployment ## Cost Estimation - **Consumption**: $0.035 per unit (1M units = ~$35) - **Developer**: $40/month (development only) - **Standard**: $306/month - **Premium**: $1224/month + $200/unit (for HA/multiple regions) ## Bicep Template ```bicep resource apimService 'Microsoft.ApiManagement/service@2023-05-01-preview' = { name: 'apim-${uniqueSuffix}' location: location sku: { name: 'Standard', capacity: 1 } identity: { type: 'SystemAssigned' } properties: { publisherName: 'Your Company' publisherEmail: 'admin@yourcompany.com' virtualNetworkType: 'Internal' customProperties: { 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls10': 'False' } } } resource openaiBackend 'Microsoft.ApiManagement/service/backends@2023-05-01-preview' = { parent: apimService name: 'openai-backend' properties: { title: 'Azure OpenAI' type: 'http' url: 'https://${openaiName}.openai.azure.com' } } ``` ## Well-Architected Framework - **Reliability**: Multiple backends, circuit breaker policies for failures - **Security**: Managed identities, key rotation via Key Vault - **Cost**: Rate limiting prevents runaway costs, caching reduces backend calls - **Performance**: Reverse proxy, response compression, caching non-LLM calls - **Operational Excellence**: Analytics on token usage, API health monitoring ## Best Practices 1. **Token-aware rate limiting** - Use policies to count tokens, not just requests 2. **Cache non-semantic requests** - Cache embeddings lookups, not completions 3. **Circuit breaker pattern** - Return cached/fallback response on timeout 4. **Monitor token spend** - Track token consumption via analytics blade 5. **Use managed identities** - Authenticate to OpenAI with service principal 6. **Implement retry logic in policy** - Handle transient failures gracefully 7. **Version APIs explicitly** - v1, v2, etc. for safe rollouts

# Key Vault Module Specification ## Overview Azure Key Vault securely stores secrets (API keys, connection strings, passwords), certificates, and encryption keys. Essential for zero-trust security. ## Key Capabilities - Secret storage and rotation - Certificate lifecycle management - Encryption key management (customer-managed encryption) - Access audit logging - Soft delete (90-day recovery) - Private endpoint support - Managed HSM (FIPS 140-3) ## Resource Inventory | Resource | Type | Purpose | |----------|------|-------| | Key Vault | Microsoft.KeyVault/vaults | Secret store | | Secrets | Secret | API keys, passwords | | Keys | Key | Customer-managed encryption keys | | Certificates | Certificate | TLS certs, code signing | | Access Policy | Policy | Granular secret permissions | ## Configuration - **SKU**: Standard (software) or Premium (HSM-backed) - **Network**: Public or Private Endpoint (VNet-isolated) - **Purge Protection**: Enable (prevents accidental deletion) - **Soft Delete**: 90 days (default, recommended) - **Access Model**: Azure RBAC (preferred) or legacy Access Policies ## Cost Estimation - **Standard SKU**: $0.03 per operation (secret read/write) - **Premium SKU**: $200/month base + operations - **Typical usage**: 1000 reads/day = ~$1/month standard - **Estimate**: $20-50/month for moderate use ## Bicep Template ```bicep resource keyVault 'Microsoft.KeyVault/vaults@2023-07-01' = { name: 'kv${uniqueSuffix}' location: location properties: { tenantId: subscription().tenantId sku: { family: 'A', name: 'standard' } accessPolicies: [] softDeleteRetentionInDays: 90 enablePurgeProtection: true enableRbacAuthorization: true publicNetworkAccess: 'Disabled' } } resource openaiSecret 'Microsoft.KeyVault/vaults/secrets@2023-07-01' = { parent: keyVault name: 'openai-api-key' properties: { value: openaiKey attributes: { enabled: true, exp: expirationDate } } } resource rbacRole 'Microsoft.Authorization/roleAssignments@2022-04-01' = { scope: keyVault name: guid(keyVault.id, principalId, 'Key Vault Secrets User') properties: { principalId: principalId roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', '4633458b-17de-408a-b874-0445c86b69e6') } } ``` ## Well-Architected Framework - **Reliability**: Soft delete enables recovery, geo-redundant storage - **Security**: RBAC access control, enable purge protection, audit all access - **Cost**: Operations-based pricing (optimize read patterns) - **Performance**: Cache secrets locally, avoid frequent lookups - **Operational Excellence**: Enable audit logging, set up alerts on suspicious access ## Best Practices 1. **Use RBAC over Access Policies** - Simpler, more granular 2. **Enable purge protection** - Prevents accidental permanent deletion 3. **Rotate secrets regularly** - Implement automation (Logic Apps, Functions) 4. **Set expiration dates** - Secrets should have TTL 5. **Audit all access** - Enable diagnostic logging, review monthly 6. **Use managed identities for access** - Never hardcode credentials 7. **Private endpoints only** - No public network access for Key Vault