Job Description

Position:

Databricks & MLOps Engineer

Description:

Design and implement Continuous Delivery (CD) pipelines for ML models using Databricks Repos, GitHub Actions, Jenkins, or Azure DevOps. Automate model deployment leveraging MLflow Model Registry, REST APIs, or Databricks Model Serving. Monitor model drift, performance, and retraining needs to ensure reliable ML solutions.

Deploy ML solutions on major cloud platforms including Azure (Databricks, AKS), AWS (SageMaker, EMR), or GCP (Vertex AI, GKE). Set up containerized ML workloads using Docker and Kubernetes.

Manage security aspects such as IAM roles and access policies across various environments. Migrate ML workflows from Airflow, Cloud Composer, or Step Functions to Databricks Jobs.

Integrate with data engineering pipelines built on Delta Lake and Spark. Track data and model lineage using Unity Catalog and MLflow.

Automate alerts for failures, performance degradation, and cost monitoring.

Design and implement scalable ML pipelines in Databricks using MLflow, Delta Lake, and Feature Store. Optimize ML model training, versioning, and deployment leveraging Databricks Jobs and Workflows.

Build reusable notebooks and libraries for model training, testing, and inference.

Implement Continuous Integration (CI) practices to improve development workflows.

Requirements:

Proven experience in CD pipeline implementation using Databricks Repos, GitHub Actions, Jenkins, or Azure DevOps.

Experience working with Airflow, Vertex AI, Docker, Kubernetes, and security management.

Strong background with Databricks, MLflow, Delta Lake, and Feature Store.

Skilled in Python, SQL, and PySpark.

Familiarity with cloud platforms including Azure, AWS, or GCP.

Experience with MLOps practices and Continuous Integration (CI) methodologies.

Benefits:

Job Tags

Similar Jobs