Job Description

We are seeking a Senior DevOps / MLOps / Data Engineer with strong experience in Azure to lead platform engineering, deployment automation, and AI/ML model deployment. This role focuses on building scalable, secure, and cost-efficient cloud solutions while enabling end-to-end ML lifecycle management and data engineering capabilities.

Key Responsibilities

DevOps, Deployment & MLOps (Primary Focus)

• Design and implement CI/CD pipelines using Azure DevOps, Git, and YAML
• Manage end-to-end deployments across Dev, QA, and Production environments
• Lead deployment of AI/ML models into production using automated pipelines
• Implement model lifecycle management (training, validation, deployment, monitoring)
• Deploy and manage batch and real-time inference models
• Automate infrastructure provisioning using ARM, Bicep, or Terraform
• Manage Azure resources via Portal, CLI, and scripting
• Oversee Databricks cluster management, scaling, and performance tuning
• Implement release strategies, versioning, rollback, and environment configuration
• Build scalable API-based model inference endpoints
• Integrate Azure AI services (e.g., Azure AI Search, Azure AI Foundry)
• Monitor system reliability, model performance, and drift

Data Engineering

• Design and develop data pipelines using Azure Data Factory, Databricks, and ADLS
• Build and optimize data models in Azure SQL, SQL Server, and Oracle
• Implement ETL/ELT processes for large-scale data processing
• Ensure data quality, governance, and performance optimization
• Support medallion architecture (Bronze, Silver, Gold layers)

Security & Compliance

• Implement secure cloud architecture using RBAC, Managed Identities, and Key Vault
• Secure data pipelines and ML endpoints (encryption, private endpoints, network controls)
• Ensure compliance with data protection and governance standards
• Manage secrets, credentials, and access policies

Cost Optimization

• Optimize cloud costs across Databricks, storage, and compute resources
• Implement cluster right-sizing and auto-scaling strategies
• Monitor usage and enforce cost governance
• Recommend cost-performance improvements

API & Integration

• Design and build scalable REST APIs for data access and model inference
• Develop API-based integrations with internal and external systems
• Enable real-time and batch data integrations
• Implement API security (authentication, throttling, versioning)
• Support event-driven architectures and messaging systems

Required Qualifications

• 10+ years of IT experience, including 5+ years in DevOps / MLOps / Data Engineering
• Strong expertise in:
  • Azure ecosystem (ADF, Databricks, ADLS, Azure SQL)
  • CI/CD pipelines, Git, and YAML
  • Infrastructure as Code (ARM, Bicep, Terraform)
  • SQL and relational databases (Azure SQL, Oracle)
  • REST API development and integration
• Proven experience in:
  • Deploying AI/ML models into production
  • End-to-end deployment pipelines and release management
  • Cluster management and optimization

Nice to Have

• Azure AI Search and Azure AI Foundry
• Event-driven architecture (Event Grid, Service Bus)
• Streaming platforms (Kafka, Event Hubs)
• Containerization (Docker, Kubernetes, AKS)
• Experience with LLMs / Generative AI pipelines
• Data governance and medallion architecture

Soft Skills

• Strong problem-solving and troubleshooting skills
• Ability to collaborate across DevOps, Data, and ML teams
• Excellent communication and documentation
• Leadership and mentoring capabilities

Must-Have Requirements

• 10+ years designing and implementing CI/CD pipelines (Azure DevOps, Git, YAML)
• 10+ years deploying AI/ML models using automated pipelines
• 10+ years implementing ML lifecycle management
• 10+ years building data pipelines using ADF, Databricks, and ADLS

Job Tags

Similar Jobs