Cloud Data Engineer

Jersey City, NJ, United States

Company Description

We help the world see new possibilities and inspire change for better tomorrows. Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, more resilient, and sustainable.

Job Description

We are looking for a Cloud Data Engineer, who will be responsible to plan, design, develop and maintain the data architecture, data models, data pipelines and standards for various Data Integration & Data Lake and Data Warehouse projects in the AWS Cloud. Ensure new features and subject areas are modelled to integrate with existing structures and provide a consistent view. Develop and maintain documentation of the data architecture, data flow and data models of the data warehouse appropriate for various audiences. Provide direction on adoption of Cloud technologies (Redshift, Postgres, Snowflake,) and industry best practices in the field of data lakes and data warehouse architecture and modelling. Providing technical leadership to large enterprise scale projects.

The role is based in our Jersey City, NJ location which has a flexible hybrid work model that has employees in-office 2-3 days per week and 2-3 days per week remote.

This role requires a broad range of skills and the ability to step into different roles depending on the size and scope of the project.

Responsibilities: -

Demonstrated ability to have successfully completed multiple, complex technical data engineering projects and create high-level design and architecture of the solution.

Able to present the solution architecture strategy and define value proposition to key stake holders.

Able to communicate the capabilities and improvements offered by solution architecture with both the development teams and business user teams.

Develop POC's and conduct show and tell sessions.

Design, build & deploy the data pipelines using EMR, EMR studio, Databricks, Spark, Pyspark and Shell Scripting.

Design the Data replication systems and integration systems by evaluating various products and deciding on the better technology that meets the corporate objectives.

Perform Data Analysis, Data Modeling, Data Management, and Data lineage.

Design data security framework considering both internal and external sharing aspects.

Design Data pipes and automate data ingestion and cleansing, transformation activities using the DevOps methodologies.

Qualifications

Must have total 7+ yrs. of experience in IT and working as a Data Engineer

Must have 5+ years of experience in EMR, Databricks, Spark, pyspark programming, Jupyter notebook and shell scripting

3+ years in Data warehouse, ETL Projects.

Must have experience End to End implementation of cloud data warehouse (Redshift, Postgres, or Snowflake).

Hands-on experience with Postgres SQL, RedShift, Redshift spectrum, AWS Glue, Athena, Snowflake utilities, Snow SQL, Snow Pipe, AWS Lambda model techniques using Python.

Experience in Data Migration from RDBMS to Redshift/Snowflake cloud data warehouse.

Expertise in data modelling, ELT using Snowflake SQL, implementing complex stored Procedures and standard DWH and ETL/ELT concepts.

Expertise in advanced concepts like setting up resource monitors, Role-based access controls (RBAC), virtual warehouse sizing, query performance tuning, zero copy clone, time travel and understand how to use these features.

Expertise in deploying features such as data sharing, events, and lake-house patterns.

Deep understanding of relational as well as NoSQL data stores, Methods, and approaches (Star and Snowflake, Dimensional Modelling).

Experience with data security and data access controls and design.

Experience with AWS or Azure data storage and management technologies such as S3 and ADLS.

Build processes supporting data transformation, data structures, metadata, dependency, and workload management.

Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot.

Provide resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface.

Must have expertise in AWS and Azure Platform as a Service (PAAS).

Should be able to troubleshoot problems across infrastructure, platform, and application domains.

Must have experience of Agile development methodologies.

Strong written communication skills. Is effective and persuasive in both written and oral communication.

#LI-LM03

#Hybrid

Additional Information

In 2022, Verisk received Great Place to Work Certification for our outstanding workplace culture for the sixth year in a row and second-time certification in the UK, Spain, and India. We’re also one of the 38 companies on the UK’s Best Workplaces list and one of 18 companies on Spain’s Best Workplaces list.

For over fifty years and through innovation, interpretation, and professional insight, Verisk has replaced uncertainty with precision to unlock opportunities that deliver significant and demonstrable impact. From our historic roots in risk assessment, we’ve grown to provide analytic insights that help transform industries focused on some of the world’s most critical areas. Today, the insurance industry relies on Verisk to be, and to make the world, more productive, resilient, and sustainable.

Verisk works in collaboration with our customers and at the intersection of people, data, and advanced technologies. Through proprietary platformed analytics, advanced modeling, and interpretation, we deliver immediate and sustained value to our customers and through them, to the individuals and societies they serve, with greater speed, precision, and scale.

We’re 9,000 people strong, committed to translating big data into big ideas. We help others see new possibilities and empower certainty into big decisions that impact individuals and societies. And we relentlessly and ethically pursue innovation to help move our customers, and the world, toward better tomorrows.

Everyone at Verisk—from our chief executive officer to our newest employee—is guided by The Verisk Way, to Be Remarkable, Add Value, and Innovate.

Be Remarkable by doing something better each day in service to our customers and each other

Add Value by delivering immediate and sustained results that drive positive outcomes

Innovate by redefining what’s possible, embracing challenges, and pushing boundaries

Verisk Businesses

Underwriting Solutions — provides underwriting and rating solutions for auto and property, general liability, and excess and surplus to assess and price risk with speed and precision

Claims Solutions — supports end-to-end claims handling with analytic and automation tools that streamline workflow, improve claims management, and support better customer experiences

Property Estimating Solutions — offers property estimation software and tools for professionals in estimating all phases of building and repair to make day-to-day workflows the most efficient

Extreme Event Solutions — provides risk modeling solutions to help individuals, businesses, and society become more resilient to extreme events.

Specialty Business Solutions — provides an integrated suite of software for full end-to-end management of insurance and reinsurance business, helping companies manage their businesses through efficiency, flexibility, and data governance

Marketing Solutions — delivers data and insights to improve the reach, timing, relevance, and compliance of every consumer engagement

Life Insurance Solutions – offers end-to-end, data insight-driven core capabilities for carriers, distribution, and direct customers across the entire policy lifecycle of life and annuities for both individual and group.

Verisk Maplecroft — provides intelligence on sustainability, resilience, and ESG, helping people, business, and societies become stronger

At Verisk you can build an exciting career with meaningful work; create positive and lasting impact on business; and find the support, coaching, and training you need to advance your career. We have received the Great Place to Work Certification for the 7th consecutive year. We’ve been recognized by Forbes as a World’s Best Employer and a Best Employer for Women, testaments to our culture of engagement and the value we place on an inclusive and diverse workforce. Verisk’s Statement on Racial Equity and Diversity supports our commitment to these values and affecting positive and lasting change in the communities where we live and work.

Verisk Analytics is an equal opportunity employer.

All members of the Verisk Analytics family of companies are equal opportunity employers. We consider all qualified applicants for employment without regard to race, religion, color, national origin, citizenship, sex, gender identity and/or expression, sexual orientation, veteran's status, age or disability.

http://www.verisk.com/careers.html

Unsolicited resumes sent to Verisk, including unsolicited resumes sent to a Verisk business mailing address, fax machine or email address, or directly to Verisk employees, will be considered Verisk property. Verisk will NOT pay a fee for any placement resulting from the receipt of an unsolicited resume.

HR CCPA Privacy Notice.pdf

#J-18808-Ljbffr