Batch scoring
Mass batch via Data Factory/Functions on Blob/Data Lake; ideal for reports, segmentations, periodic enrichment.
AI Integration
Reference architectures and patterns to bring Azure AI into enterprise apps and processes.
Technology Cluster · Back to Azure AI Technologies · Azure ML
Integration patterns
Real‑time inference
Online endpoints (AKS/ACI) for interactive experiences: recommendations, fraud detection, assistants.
Event‑driven & streaming
Event Grid/Hub + Stream Analytics/Functions for low‑latency pipelines and controlled back‑pressure.
Retrieval‑augmented (RAG)
Enrich LLM with enterprise sources (index + embeddings) for cite‑back, up‑to‑date answers.
MLOps & lifecycle
Registry, environments, CI/CD, monitoring and retraining; clear roles and environments separation.
Security & compliance
Private networks, managed identity, key vault, encryption, logging, DLP and retention policies.
Reference architecture
Components
- Data Lake / Feature Store
- Azure ML (train/deploy) + Endpoints
- App integration (API/SDK, Service Bus, Functions)
- Monitoring (Application Insights, Log Analytics)
Flows
Ingest → Storage → Features → Train → Registry → Deploy → Monitor → RetrainAutomate with pipelines and quality gates.
Pattern selection
| Pattern | Latency | Volume | Examples |
|---|---|---|---|
| Batch | High (minutes‑hours) | Very high | Reports, nightly scoring |
| Real‑time | Low (ms‑s) | Medium | In‑session recommendations |
| Event‑driven | Low (ms‑s) | High | IoT, logs, clickstream |
| RAG | Medium | Variable | Knowledge & assistants |
Best practices
Observability
Track requests, queues, timings, errors and costs; define SLOs and alerts.
Reliability
Retry/backoff, idempotency, circuit breakers and rate limiting on APIs.
Change management
Model/API versioning, canary/blue‑green, compatibility and rollback plan.
Next steps
FAQ
How to handle traffic spikes?
Autoscaling, async queues and caching; separate compute planes to avoid contention.
Which KPIs should I track?
Latency P50/P95, error rate, throughput, cost per call, model quality (F1/AUC), drift.
How to protect secrets?
Key Vault, managed identity, secret rotation and least‑privilege policies.