Big Data Hadoop Engineer

Pleasanton, CA, United States

Title: Big Data Hadoop Engineer

Location: Pleasanton, CA

Duration: 12 months+

Compensation: $52/hr - $74/hr

Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S.

ABOUT THE ROLE:

The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long-term IT strategy and planning to include high-level analysis, professional reports and presentations, and mentoring, support and training.

CURRENT SITUATION:

We are looking for consultant who can provide vision and technical leadership on the project. The resource will review the existing the architecture and recommend new technologies as appropriate. Hadoop engineer will closely work with business units and recommend changes to provide value to business.

TASKS:

Provide vision, gather requirements and translate client user requirements into technical architecture.

Design and implement an integrated Big Data platform and analytics solution

Design and implement data collectors to collect and transport data to the Big Data Platform.

Implement monitoring solution(s) for the Big Data platform to monitor health on the infrastructure

RESOURCE REQUIREMENTS, SKILLS, KNOWLEDGE AND ABILITIES:

Resource assigned to the project have the minimum skills requirement to render the services in a competent and efficient manner.

Technical Knowledge and Skills:

Strong project experience in Big Data, Cloudera Distribution 7.x, Cloud migration, RDBMS is must.

Strong project experience with Amazon EMR/Databricks/Cloudera CDP is must.

4-5 years of programming experience in Python , Java and Scala is must.

Strong experience with REST API development using Python frameworks ( Django , Flask etc.)

Micro Services/Web service development experience using Spring framework

Strong experience in Amazon EMR/DataBricks/Cloudera CDP

Experience with Dask , Numpy , Pandas , Scikit-Learn

Hands-on experience in Big Data , Cloudera Distribution 7.x, RDBMS

Hands-on experience building data pipelines using Hadoop components Sqoop , Hive , Solr , MR , Impala , Spark , Spark SQL ., HBase .

Strong experience working in Real-Time analytics like Spark / Kafka / Storm

Experience with Jenkins , JIRA

Expertise in Unix / Linux environment in writing scripts and schedule/execute jobs.

PROFESSIONAL SKILLS: Strong analytical skills with the ability to analyze information and identify and formulate solutions to problems.

Provide more in-depth analysis with a high-level view of goals and end deliverables.

Complete work within a reasonable time frame under the supervision of a manager or team lead.

Plan and manage all aspects of the support function.

Extensive knowledge of and proven experience with data processing systems, and methods of developing, testing and moving solutions to implementation.

Strong knowledge in project management practices and ability to document processes and procedures as needed.

Work collaboratively with other support team members and independently on assigned tasks and deliverables with minimum supervision

Communicate effectively with users at all levels, from data entry technicians up to senior management, verbally and in writing.

Self-motivated, working closely and actively communicating with team members to accomplish time critical tasks and deliverables

Ask questions and share information gained with other support team members, recording and documenting this knowledge

Elicit and gather user requirements and/or problem description information, and record this information accurately

Listen carefully and act upon user requirements

Convey and explain complex problems and solutions in an understandable language to both technical and non-technical persons

Present technical solutions to management and decision makers

Follow the lead of others on assigned projects as well as take the lead when deemed appropriate

Think creatively and critically, analyzing complex problems, weighing multiple solutions, and carefully selecting solutions appropriate to the business needs, project scope, and available resources

Take responsibility for the integrity of the solution.

Must Haves- Strong project experience in Big Data, Cloudera Distribution 7.x, Cloud migration, RDBMS is must.

Strong project experience with Amazon EMR/Databricks/Cloudera CDP is must.

4-5 experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL., HBase.

4-5 years of programming experience in Python, Java and Scala is must.

Strong experience with REST API development using Python frameworks (Django, Flask etc.)

Micro Services/Web service development experience using Spring framework is highly desirable

CORE COMPETENCIES: Act with integrity

Use sound judgement

Commitment to quality

Demonstrate adaptability

Innovate

Think strategically

Communicate effectively and influence others

Work well both independently and as part of a team

Our benefits package includes: Comprehensive medical benefits

Competitive pay

401(k) retirement plan

...and much more!

About INSPYR Solutions

Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.

INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.

24-02720