Senior Azure Databricks Platform Engineer

Prodapt is the largest specialized player in the Connectedness industry. As an AI-first strategic technology partner, Prodapt provides consulting, business reengineering, and managed services for the largest telecom and tech enterprises building networks and digital experiences of tomorrow. A ServiceNow-invested company, Prodapt has been recognized by Gartner as a Large, Telecom-Native, Regional IT Service Provider. A “Great Place To Work® Certified™” company, Prodapt employs over 6,000 technology and domain experts across the Americas, Europe, India, Africa, & Japan. Prodapt is part of the 130-year-old business conglomerate The Jhaver Group, which employs over 32,000 people across 80+ locations globally. We are seeking a highly skilled Senior Azure Databricks Platform Engineer to lead the deployment, migration, optimization, and governance of enterprise-scale Databricks platforms on Azure. The ideal candidate brings deep expertise in Databricks architecture, Azure cloud services, data platform administration, and production support, with a strong focus on performance, scalability, reliability, and automation. KEY RESPONSIBILITIES

Prepare and configure the Databricks Serverless environment for enterprise teams, ensuring all required libraries, dependencies, and platform components are available and maintained. Design, deploy, and manage Databricks job clusters and workflows. Collaborate with Databricks and enterprise stakeholders to enable and optimize AI/ML capabilities across the platform. Migrate existing Databricks jobs, workflows, and data processing pipelines to modernized environments. Troubleshoot and resolve Azure networking, proxy, firewall, authentication, and connectivity issues affecting Databricks and related applications. Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency. Enhance operational monitoring and observability by building solutions that track job execution status, start/end times, duration, failures, and overall health. Maintain and enhance the DataSphere health check framework and associated automation processes. Implement and support Unity Catalog, access controls, governance policies, and security best practices. Investigate and resolve production incidents, failed jobs, and data pipeline issues. Create and manage Databricks Genie Spaces for business domains as part of migration and modernization initiatives.

REQUIRED QUALIFICATIONS

10+ years in Data Engineering / Cloud Data Platforms, with strong Azure Databricks administration and migration expertise Bachelor's or Master's degree in Computer Engineering, or in a related field. Strong hands-on experience with Azure Databricks, including: Workflows, jobs, and cluster management Serverless compute Performance tuning and optimization Proven experience with Microsoft Azure Cloud, including: Virtual networking Firewalls and proxy configurations Authentication and authorization Strong programming skills in Python and SQL. Experience with ETL/ELT development, migration projects, and modern data engineering practices, including direct Databricks and Snowflake platform experience. Expertise in production support, monitoring, alerting, incident management, and job reliability improvements. Experience with Unity Catalog, data governance, access management, and data quality frameworks.

PREFERRED QUALIFICATIONS

Experience with Databricks AI/ML capabilities and generative AI initiatives, including LLM-powered applications and AI-driven workflows. Experience building operational dashboards and health-check automation frameworks. Knowledge of SSO, OAuth, Microsoft Entra ID, and identity management solutions. Experience developing platform monitoring, observability, and operational support portals. Familiarity with enterprise-scale cloud data platforms and governance frameworks.

DESIRED SKILLS Databricks | Azure Cloud | Python | SQL | Snowflake | Unity Catalog | ETL/ELT | Data Governance | Monitoring & Observability | React | TypeScript | Azure Web Apps | AI/LLM | Entra ID | Production Support

#J-18808-Ljbffr