Lead Data Engineer
San Francisco, CA, United States
Company
At Directly, we believe the fundamental nature of work is changing from fixed to fluid, as AI and the gig economy transform how people and companies work in the next economy. We’re leading this shift with an enterprise platform that changes the way companies deliver customer service. Our platform helps companies like Airbnb, Autodesk, LinkedIn, Microsoft, Intuit, SAP, and Samsung look beyond the traditional contact center and deliver better answers to customers in the moments that matter. We’re based in San Francisco and backed by top firms including Microsoft Ventures, True Ventures, Costanoa Ventures, and Northgate. www.directly.com
Role
We are looking for new members of our Data Science and Engineering team to help build augmented intelligence systems that allow humans and machines to work together to redefine customer support.
Our work spans many disciplines, such as information retrieval, natural language processing, and machine learning. Directly was granted several patents on our innovative technology, and we are constantly expanding the science and inventing new processes for optimizing the benefits of our platform.
Responsibilities
Own and evolve the core of the data pipeline and lifecycle at Directly, whether that involves growing the data model and schema, or implementing systems for data quality and furthering data management through ETL, or improving the efficiency of data ingest.
Analyze performance of data products and propose architectural improvements.
Work as part of Engineering team to ship new features of data products into production.
Architect the ETL for our growing data set and user needs and plan its roadmap.
Implement SQL queries for RedShift, and/or a Hadoop job on EMR; and define and implement computational tasks on Spark and other High-Performance platforms.
Work with the DevOps team on defining the infrastructure requirements for data products to operate reliably at scale, serving analytics and machine learning, and other use cases.
Requirements
Academic background in a technical/quantitative field or equivalent experience.
Strong software engineering skills. You understand how to architect, develop, and ship maintainable code in a group, follow a process, test, document, and support the product.
Data modeling and ETL experience; you have architected and built an enterprise grade Data Warehouse supporting multiple stakeholders (analytics, data science, etc.).
Knowledge of best practices, processes and tools of software development in a team.
Experience with a variety of High-Performance computational frameworks (Spark, Hadoop, Elasticsearch, Kafka, AirFlow, etc.).
Fluency in computer languages used in Data projects (Python, Java, Scala, Shell, etc.).
Experience with deploying solutions in AWS.
Experience building large distributed systems and data pipelines.
Experience with technical leadership of a team and/or growing a team.
Directly is an equal opportunity employer. We believe in diversity and inclusion as a strength.
#J-18808-Ljbffr