Sr. Data Engineer with Unity Catalog

Overview

Seattle-based client — Location: Seattle, WA. Duration: 12 months. 4 days in-office. Core Skills include data governance with Databricks Unity Catalog. 10-12+ years of experience in data engineering or a related field. Must coordinate with Offshore teams.

Responsibilities

• Design, build, and deploy data extraction, transformation, and loading processes and pipelines from various sources including databases, APIs, and data files.
• Develop and support data pipelines within a Cloud Data Platform, such as Databricks.
• Build data models that reflect domain expertise, meet current business needs, and remain flexible as strategy evolves.
• Monitor and optimize Databricks cluster performance, ensuring cost-effective scaling and resource utilization.
• Communicate technical concepts to non-technical audiences in both written and verbal form.
• Implement and maintain Delta Lake for optimized data storage, ensuring data reliability, performance, and versioning.
• Automate CI/CD pipelines for data workflows using Azure DevOps.

Qualifications

• Experience in a cloud environment (Azure preferred) with strong understanding of cloud data architecture.
• Hands-on experience with Databricks Cloud Data Platforms.
• Experience migrating to Unity Catalog.
• Experience with workflow orchestration (e.g., Databricks Jobs, Azure Data Factory pipelines).
• Programming languages: Python/PySpark, SQL, or Scala.

Requirements

• Location: Seattle, WA (on-site/off-site balance as specified).
• Employment type: Full-time.
• Seniority level: Mid-Senior level.
• Industry: IT Services and IT Consulting.