Senior Data Platform Engineer

Design, build, and enhance platform capabilities within Databricks and related technologies Evaluate and adopt emerging platform features and technologies Run self-assessments and technology evaluations on which research initiatives depend Improve governance, automation, observability, security, and operational excellence Help define the target-state architecture for the custom platform Use AI agents and agentic tools to design, build, test, and operate systems, verifying outputs appropriately Build reusable platform capabilities and data products for quantitative researchers to discover, access, and consume data Accelerate successful prototypes to production Support and advise Giant Tech customers on Microsoft and other industry-leading technologies Requirements Hands-on Databricks experience across workspace, notebooks, and jobs, plus building platform capabilities on it (not just using it) Strong Python and SQL Infrastructure as code with Terraform and modern CI/CD Data governance and observability in a managed environment, such as Unity Catalog, lineage, and monitoring Strong workflow orchestration experience with Apache Airflow; the company uses Astro / Astronomer Experience building and operating data pipelines integrating disparate, heterogeneous data sources Experience building reusable, self-service data products for non-engineer end users, including researchers and analysts AI-first mindset; fluency with agentic coding tools such as Claude Code Full-time availability Core Competencies Demonstrates expertise in Databricks platform capabilities, Python, SQL, and Terraform for building and enhancing data products. Proficient in data governance, observability, and workflow orchestration to support quantitative research initiatives. Highest-signal resume keywords Databricks Experience Python Programming SQL Proficiency Infrastructure As Code (Terraform) Workflow Orchestration (Apache Airflow) Hard Skills Databricks Python SQL Terraform CI/CD Data Governance Observability Apache Airflow Data Pipeline Development AI Agentic Tools Industry Keywords Data Products Quantitative Research Operational Excellence Automation Security Tools & Technologies Unity Catalog Astro Astronomer

#J-18808-Ljbffr