Job Description

Job Location: Triangle Area, NC (Hybrid)

Note: Candidate must be local to the Triangle region of North Carolina. Onsite presence is required on the first day to collect equipment and up to 1–2 days/month for in-person meetings.

• Each candidate must submit a government-issued ID (Driver’s License or Passport) and provide three professional references (names, official emails, and phone numbers).

Job Description:

The North Carolina Department of Information Technology – Transportation (NCDOT) seeks an Azure Databricks Engineer to support the Web Systems Team in planning and designing scalable ETL pipelines using Azure Databricks. This position focuses on creating resilient workflows to ingest and process data from various on-premise and cloud-based sources, ensuring robust data architecture and product delivery for NCDOT analysts and stakeholders.

Specialty Areas:

• Azure Databricks Engineering – Designing and implementing Delta Lake architecture using Delta Live Tables and Spark.

• ETL & Data Pipeline Development – Creating batch and streaming pipelines across cloud and hybrid data platforms.

• Data Architecture & Quality – Building secure, well-documented, and validated pipelines aligned with enterprise data standards.

• Stakeholder Collaboration – Working closely with development leads and business users to develop high-impact data products.

Responsibilities:

• Engineer robust ETL workflows using Databricks notebooks and Delta Live Tables for batch and stream processing.

• Develop data products aligned with business domain needs and collaborate with DBAs for data ingestion.

• Support secure data development practices and ensure sensitive data protection.

• Maintain documentation for all processes and implement data validation/testing procedures.

• Share architectural insights and collaborate with data teams across the organization.

Skill Matrix:

Skill
Experience (Years)

2+ (Required)

2+ (Required)

2+ (Required)

2+ (Required)

2+ (Required)

2+ (Required)

7+ (Desired)

5+ (Required)

Mandatory Requirements:

• Experience in Databricks and Spark development within Azure or similar cloud platforms.

• Proven ability to build and document scalable ETL pipelines.

• Strong communication skills and collaborative mindset.

• Must be a North Carolina local, available for occasional in-person meetings.

Preferred Qualifications:

• Experience with multiple object-oriented programming languages.

• Prior work with public sector data systems or government agencies.

Submission Requirements:

• Updated Résumé

• Government-issued ID (Driver’s License or Passport)

• Three professional references (Names, official emails, phone numbers)

Job Tags

Similar Jobs