AI Devops Engineer - Infrastructure Engineer III

AI DevOps Engineer - Deloitte Support Services India Private Limited

Work you’ll do:

As a professional in TAX to design, automate, secure, and operate scalable platforms for artificial intelligence and machine learning workloads.

This hands-on role combines cloud engineering, DevOps, Kubernetes, infrastructure as code, MLOps, and LLMOps. The engineer will support the full AI lifecycle—from experimentation and model packaging through deployment, monitoring, optimization, retraining, and retirement.

The primary cloud environment is Microsoft Azure, with additional experience in AWS, including Amazon Bedrock, highly valued. The successful candidate will partner with data scientists, ML engineers, software engineers, security teams, and business stakeholders to deliver reliable, production-grade AI solutions.

Key Responsibilities

AI/ML Platform Engineering

    - Design and operate secure, scalable, and highly available platforms for machine learning, generative AI, and LLM workloads.

    - Develop standardized deployment patterns for traditional ML models, deep learning models, LLMs, RAG applications, and AI agents.

    - Support real-time, batch, asynchronous, and event-driven model-serving architectures.

    - Deploy and manage model endpoints, embedding services, prompt-processing services, retrieval pipelines, and inference APIs.

    - Optimize AI platforms for performance, availability, scalability, resilience, and cost.

    - Partner with data science and ML engineering teams to productionize experimental models.

    - Establish repeatable processes for model packaging, validation, deployment, rollback, versioning, and retirement.

CI/CD and DevSecOps Automation

    - Design and maintain CI/CD pipelines using GitHub Actions and Azure DevOps.

    - Automate deployments for application code, infrastructure, container images, model artifacts, and ML workflows.

    - Implement automated testing for code, infrastructure, data pipelines, model packages, security controls, and deployment configurations.

    - Establish promotion workflows across development, test, staging, and production environments.

    - Automate model validation, deployment-readiness checks, approval gates, rollback, canary releases, and controlled production promotion.

    - Integrate source control, artifact repositories, container registries, model registries, cloud services, and observability platforms.

    - Apply GitOps, pull-request-based deployment, reusable workflow templates, and environment protection practices.

Containers and Kubernetes

    - Design, deploy, and operate production-grade Kubernetes and Azure Container Instances environments for AI and ML workloads.

    - Build secure, efficient Docker images for model-serving APIs, data-processing jobs, feature services, and LLM applications.

    - Manage Kubernetes deployments, services, ingress, configuration, secrets, storage, namespaces, resource quotas, and autoscaling.

    - Implement workload scheduling for CPU-, GPU-, and memory-intensive applications.

    - Use Helm, Kustomize, Kubernetes operators, and platform integrations to support model serving, distributed training, experiment tracking, and batch inference.

    - Troubleshoot production issues involving containers, networking, storage, scheduling, performance, and resource utilization.

    - Establish standards for cluster security, upgrades, capacity planning, backup, and disaster recovery.

Cloud Engineering and Infrastructure as Code

    - Develop reusable Terraform modules and Azure Bicep templates for multi-environment cloud infrastructure.

    - Build secure cloud foundations using identity and access management, private networking, encryption, secrets management, policy enforcement, and centralized logging.

    - Automate infrastructure provisioning, validation, drift detection, controlled changes, and rollback.

    - Integrate infrastructure deployment with CI/CD pipelines and approval workflows.

    - Monitor and optimize cloud costs, including compute, storage, data transfer, GPU capacity, and AI inference usage.

    - Support high availability, business continuity, disaster recovery, and regional resiliency requirements.

MLOps and Model Lifecycle Management

    - Implement MLOps practices covering experimentation, training, validation, registry management, deployment, monitoring, retraining, and retirement.

    - Manage model registries and experiment-tracking platforms such as MLflow, Kubeflow, or comparable tools.

    - Govern model versions, metadata, lineage, ownership, approval status, and deployment history.

    - Establish reproducibility across code, data, model artifacts, dependencies, configurations, and runtime environments.

    - Implement model rollback, challenger models, A/B testing, canary deployments, shadow deployments, and automated model-refresh workflows.

    - Monitor model, data, and concept drift and support remediation through retraining or controlled model updates.

LLMOps and Generative AI Operations

    - Deploy and operate hosted and self-managed LLMs in secure, production-grade environments.

    - Build deployment workflows for prompt templates, embedding models, rerankers, retrieval services, and AI application components.

    - Support RAG, semantic search, agentic workflows, tool-using applications, and vector database integrations.

    - Manage model configuration, context windows, token limits, concurrency, batching, caching, rate limits, quotas, and RPM/TPM usage.

    - Optimize model selection, latency, throughput, response quality, token consumption, and overall GenAI cost.

    - Implement controls for sensitive-data exposure, prompt injection, jailbreaks, unauthorized access, unsafe responses, and data leakage.

    - Apply GenAI SDKs and orchestration frameworks such as OpenAI/Azure AI SDKs, Google ADK, LangChain, and LangGraph.

Observability, Reliability, and Incident Management

    - Implement observability across AI platforms, infrastructure, applications, models, inference endpoints, retrieval systems, and vector databases.

    - Monitor latency, throughput, concurrency, errors, timeouts, token usage, GPU/CPU utilization, memory, queue depth, availability, model quality, and drift.

    - Use tools such as Prometheus, Grafana, OpenTelemetry, Azure Monitor, CloudWatch, Langfuse, LangSmith, or comparable platforms.

    - Create dashboards, alerts, health checks, runbooks, service-level objectives, and escalation procedures.

    - Implement distributed tracing and correlation across AI application components.

    - Lead incident response, root-cause analysis, remediation, and post-incident improvement activities.

    - Design reliability patterns such as graceful degradation, fallback models, retry logic, circuit breakers, and disaster recovery.

Security, Governance, and Responsible AI

    - Embed security controls throughout the AI, cloud, infrastructure, and software delivery lifecycle.

    - Implement least-privilege access, identity federation, secrets management, network segmentation, encryption, and secure service-to-service communication.

    - Integrate source-code, dependency, container-image, infrastructure, and model-artifact scanning into CI/CD.

    - Apply policy-as-code and automated compliance checks.

    - Maintain auditability for infrastructure changes, model releases, deployments, access events, and incidents.

    - Partner with security, privacy, risk, and compliance teams to support responsible AI operations.

    - Promote traceability, human oversight, access control, safety evaluations, explainability, and controlled model changes.

Technical Leadership and Collaboration

    - Define reference architectures, platform standards, reusable components, and technical roadmaps.

    - Lead architecture reviews, technical design sessions, code reviews, and production-readiness assessments.

    - Mentor engineers and promote effective DevOps, SRE, cloud, and MLOps practices.

    - Translate business and AI product requirements into secure, scalable, supportable solutions.

    - Communicate technical risks, dependencies, delivery status, and trade-offs to engineering and business leadership.

    - Create technical documentation, operating procedures, runbooks, and knowledge-transfer materials.

Preferred Qualifications

    - Experience operating AI workloads across multiple cloud platforms.

    - Experience with Azure Bicep, policy as code, and secure cloud landing zones.

    - Experience with Pinecone, Milvus, or other vector databases.

    - Experience with KServe, Seldon, NVIDIA Triton, BentoML, TorchServe, or comparable model-serving technologies.

    - Experience with GPU scheduling, CUDA-enabled workloads, quantization, batching, and inference optimization.

    - Experience with LLMOps, RAG architectures, prompt lifecycle management, AI agents, and LLM evaluation.

    - Familiarity with Azure OpenAI, Azure AI Foundry, Amazon Bedrock, Amazon SageMaker, or Google Vertex AI.

    - Experience with responsible AI, model risk management, privacy, or regulatory controls.

    - Relevant certifications such as CKA, Terraform Associate, AWS Certified Machine Learning Engineer, Azure AI Engineer Associate, Google Professional Machine Learning Engineer, or comparable credentials.

Representative Technology Environment

Cloud:

Azure, AKS, Azure Machine Learning, Azure OpenAI, Azure AI Foundry, AWS, Amazon Bedrock, EKS

DevOps:

GitHub Actions, Azure DevOps, GitOps, artifact promotion

Infrastructure:

Terraform, Bicep, policy as code

Containers:

Docker, Kubernetes, Helm, Kustomize, ACI

MLOps:

MLflow, Kubeflow, KServe, Seldon, Triton, BentoML

GenAI:

LangChain, LangGraph, RAG, embeddings, vector databases, prompt management

Observability:

Prometheus, Grafana, OpenTelemetry, Azure Monitor, CloudWatch, Langfuse, LangSmith

Security:

IAM, Key Vault, Trivy, Snyk, SonarQube, Checkov, GitHub Advanced Security

Education:

·       Bachelor’s or master’s degree in computer science, Information Technology, or a related field.

Experience:

• 5-8 years of experience in Azure DEVOPS
• Proven experience in leading and managing Azure DEVOPS projects.
• Strong leadership and team management skills.
• Should be AZ-104 and AZ-400 certified, AWS cloud practitioner or Google Cloud Professional is a plus.

Location: Hyderabad 

Shift Timing: 09:00 AM - 06:00 PM IST

                     : 11:00 AM - 08:00 PM IST

                     : 02:00 PM - 11:00 PM IST

The team

At Deloitte, we’re all about collaboration. And nowhere is this more apparent than among our 2,000-strong internal services team. With our combined specialist skills, we provide all the essential support and advice our client-facing colleagues need, right across the firm. This enables them to focus all of their efforts on delivering the best service possible to their clients. Covering seven distinct areas; Human Resources, Clients & Industries, Finance & Legal, Practice Support Services, Quality & Risk Services, IT Services, and Workplace Services & Real Estate, together we live, breathe and deliver the Deloitte experience.

How you’ll grow

At Deloitte, we’ve invested a great deal to create a rich environment in which our professionals can grow. We want all our people to develop in their own way, playing to their own strengths as they hone their leadership skills. And, as a part of our efforts, we provide our professionals with a variety of learning and networking opportunities—including exposure to leaders, sponsors, coaches, and challenging assignments—to help accelerate their careers along the way. No two people learn in exactly the same way. So, we provide a range of resources including live classrooms, team- based learning, and eLearning. DU: The Leadership Center in India, our state-of-the-art, world-class learning Center in the Hyderabad offices is an extension of the Deloitte University (DU) in Westlake, Texas, and represents a tangible symbol of our commitment to our people’s growth and development. Explore DU: The Leadership Center in India

Benefits

At Deloitte, we know that great people make a great organization. We value our people and offer employees a broad range of benefits. Learn more about what working at Deloitte can mean for you.

Deloitte’s culture

Our positive and supportive culture encourages our people to do their best work every day. We celebrate individuals by recognizing their uniqueness and offering them the flexibility to make daily choices that can help them to be healthy, centered, confident, and aware. We offer well-being programs and are continuously looking for new ways to maintain a culture that is inclusive, invites authenticity, leverages our diversity, and where our people excel and lead healthy, happy lives. Learn more about Life at Deloitte.

Corporate citizenship

Deloitte is led by a purpose: to make an impact that matters. This purpose defines who we are and extends to relationships with our clients, our people and our communities. We believe that business has the power to inspire and transform. We focus on education, giving, skill-based volunteerism, and leadership to help drive positive social impact in our communities. Learn more about Deloitte’s impact on the world.