Data Engineer

Staff Engineer The Staff Engineer will play a pivotal role in building and operationalizing the enterprise data and analytics initiatives following industry standard practices and tools. This resource will be responsible for building, managing and optimizing data pipelines and automating production deployments. This is a technical leadership role working closely with business and IT partners to solve business problems using data engineering and analytics.

Principal Responsibilities:

Designs and develops code and data pipelines to ingest from relational databases (Oracle, SQL Server, DB2, Aurora), file shares, and web services.

Streaming ingestion with Kinesis Streams, Kinesis Firehose, Kinesis Analytics and Kafka(MSK)

Build Data Lake on AWS S3 with optimal performance considerations by partitioning and compressing data.

Data Engineering and Analytics using AWS Glue, Informatica, EMR, Spark, Athena, Python, PowerBI and Sagemaker.

Data modeling and building Data Warehouse using Snowflake.

Participates in requirements definition, system architecture design, and data architecture design.

Participates in all aspects of the software life cycle using Agile development methodologies.

Qualifications:

8+ years of experience in Enterprise Information Solution Architecture, Design, and development required.

8+ years of hands-on experience with AWS services (S3, Kinesis, Lambda, Athena, Glue, EMR) required.

8+ years of experience with integration architectures such as ETL, SOA, Microservices or other integration technologies.

5+ years of Web development experience with Python and Node.js.

3+ years of experience building cloud data warehouse using Snowflake and/or Redshift.

3+ years of NoSQL database DynamoDB, Marklogic, MangoDB.

3+ years of Business Intelligence applications like Qlik Sense, Tableau, or PowerBI.

3+ years of analytics tools like SAS, R, Python, and other advanced statistical software.

Experience with Machine Learning and Artificial Intelligence.

Experience with DevOps and Terraform.

Ability to multi-task effectively.

Ability to work collaboratively as part of an Agile Team.

Knowledge of multiple programming environments, debugging tools, and test tools required.

Domain knowledge of NoSQL or relational database required.

Experience developing enterprise grade production software required.

Understanding of database architecture and performance implications required.

Excellent written and verbal communication skills, sense of ownership, urgency and drive.