Job Description

Role: Databricks Data Engineer

Location: Remote

Type: Contract

Job Description:

• We are seeking an experienced Databricks Data Engineer to join our team for a 6+ month remote contract engagement.
• The ideal candidate will have a strong background in big data engineering, cloud data platforms, and distributed data processing, with a particular focus on Databricks and the Azure ecosystem.

Key Responsibilities:

• Design, develop, and optimize scalable data pipelines and data lakes using Databricks and Apache Spark.
• Implement ETL/ELT processes to ingest, transform, and store structured and unstructured data from various sources.
• Work with large-scale datasets to support advanced analytics and machine learning workloads.
• Collaborate with Data Scientists, Data Analysts, and other stakeholders to understand data requirements and deliver optimal solutions.
• Monitor, troubleshoot, and ensure data pipeline performance, reliability, and data quality.
• Build reusable components and frameworks for data engineering tasks in Azure Databricks.
• Document technical processes, data flows, and solution architecture.

Required Skills and Qualifications:

• 5+ years of experience in Data Engineering or related fields.
• 2+ years hands-on experience with Databricks (including notebooks, Delta Lake, and Spark jobs).
• Strong proficiency in Apache Spark (Scala or PySpark).
• Solid experience with Azure data services (Azure Data Lake, Azure Data Factory, Azure Synapse, etc.).
• Strong SQL and data modeling skills.
• Proficient in Python or Scala scripting for data processing.
• Experience with CI/CD pipelines for data engineering workflows (Azure DevOps preferred).
• Understanding of data governance, security, and best practices in cloud data platforms.
• Excellent problem-solving skills and ability to work in a fast-paced environment.

Preferred Qualifications:

• Databricks or Microsoft Azure certifications.
• Experience with real-time data processing and event streaming (e.g., Kafka, Event Hubs).
• Familiarity with Delta Live Tables and Unity Catalog.
• Experience working in Agile/Scrum environments.

Job Tags

Similar Jobs