Big Data Engineer

GCP Senior Big Data EngineerLooking for a Senior Big Data Engineer/Architect on Google Cloud Platform to help strategize, architect and implement various solutions to migrate data and pipelines hosted on our on-prem big data platform to Google cloud Platform (GCP)/DataProc. Our team builds out the wireless personalization framework. This framework uses AI/ML to score the customers experience on Verizons wireless network cell tower network. You will be exposed to 1) Big Data at the Verizon scale (100s of TBs / day). 2) Data Wrangling, Cleansing and Correlation and 3) Machine Learning algorithms and 4) High performance serving infrastructure for both internal business partners as well as our customer base.As a GCP Big Data Engineer - You should be comfortable with Big Data Technologies like Hadoop, Hive, Apache Spark, Oozie and Google Cloud platform. You will work on advanced Apache Spark/Oozie pipelines on our on-prem hadoop platforms as well as GCP Dataproc environment and helping strategize the migration to GCP.Required:Experience in Design, construct, test, tune, and deploy ETL infrastructure based on the Hadoop ecosystem.5+ years of hands-on experience in development using Spark/pyspark, HDFS, HIVE, Oozie.Strong understanding about Google Cloud Platform Technologies, GCP DataProc.Must have strong programming experience with languages like Python, Java.Experience with UNIX/Linux systemsBuilding orchestration and processing flows using Oozie.Experience in translating functional and technical requirements into technical specifications and design.Good experience with Big data ingestion, data processing and data modelling processStrong knowledge of data structures, algorithms, enterprise systems, and asynchronous architecturesExperience in taking proactive actions to Ability to identify typical problems and issues during the normal course of work and solve them with minimum guidanceGood to have:Experience or knowledge with MapReduce, Yarn, Tez, Zookeeper and YarnProficiencies with Agile development practices and CICD process..Hands-on background with both noSQL databases such as Hbase/Druid/MongoDB and relational database such as Oracle/MySql.