Big Data Engineer

Software Engineer Tech LeadAs a Software Engineer Tech Lead, you will be responsible for leading a team of software engineers in an initiative to migrate an on-premises large-scale data warehouse and associated data pipelines to Azure. You will have core competencies in the big data analytics platforms including Spark/Databricks as well as Data Warehouse/Data Lakehouse architectures. You will also have very strong project management skills, specifically using Agile Methodologies such as Scrum/Kanban/Lean.Key Responsibilities:Lead a team of software engineers in the migration of a large-scale data warehouse and associated data pipelines.Ability to have trade-off discussions, with engineering team, around data pipeline design with the goal of driving consensus on approach.Attend cross-functional meetings with project Technical Product Manager (TPM).Work with the TPM to translate requirements into backlog itemsLead the engineering team in backlog grooming and refinement meetings.Work with team to prioritize product and engineering backlog items in planning meetings.Lead daily stand-up meetings and drive action items and ownership around blocking items.Communicate schedule/project timeline status to engineering manager and TPM, highlighting key trade-off discussions.Mentor junior engineers and help them to grow their skills in story creation, decomposition, and estimation.Qualifications:7+ years of experience in software engineering and data engineeringExperience with Azure Cloud Analytics Platforms including PaaS and IaaS services: Azure Data Lake Storage, Azure Data Factory, Kafka, Azure SynapseExperience with Analytics Platforms and related technologies: Spark/Databricks, Apache Streaming, Delta Tables, and AirflowExperience leading a team of software engineers in large-scale data warehouse and data pipelines migration.Experience designing and implementing data pipelines and workflow managementExperience with data warehouse/lake house architectures and data modelingStrong problem-solving skillsStrong communication and leadership skillsStrong ability to drive engineering trade-off discussions with the goal of driving consensusExperience in Agile methodologies such as Scrum, specifically in the role of Scrum MasterBachelor's degree in Computer Science or related field.