Lead Big Data Developer
Rockville, MD, United States
Position: Lead Big Data Developer
Location: Rockville, MD (onsite once a week)
Length: Long Term Contract
Term : Contract (C2C)
Please apply if you have completed at least three projects as a Lead Big Data Developer real-time experience. Background and employment verification will be conducted.
Interview Process: Candidates who pass the written test will proceed to three rounds of technical interviews conducted via video call, which will include live coding scenarios.
If you are interested, please send your resume to hr@.
Job Description:
B.I.R Solutions is seeking a highly motivated Lead Big Data Developer to support our client. The ideal candidate need to meet the following criteria:
Requirements:
10+ years of development experience in Java AND Scala working on ETL projects in an Agile environment
10+ years of experience in big data projects
10+ years of experience with API development (REST API's) and processing with JSON and csv files
10+ Years of experience working in a Lead role supporting design of applications and ETL projects
Strong knowledge of big data technologies including: AWS EMR, AWS EKS, Apache Spark, SQL & serverless technologies like AWS Dynamo DB, AWS Lambda
Unit testing using JUnit or ScalaTest
Desired:
Business Process Management experience using AWS functions
Python & Shell Scripting
Experience with performance testing tools such as Gatling or Jmeter
Responsibilities:
Comprehend Complex Business Requirements
Design and develop ETL pipelines to collect, validate, and transform data according to the requirement
Develop automated unit tests, functional tests and performance tests.
Support optimal data pipeline architecture
Design ETL jobs for seamless execution in AWS cloud environment
Reduce processing time and cost of ETL workloads
Lead cross-functional review meetings with peers and stakeholders
Continuous data quality checks.
Identify areas where machine learning can be used to identify data anomalies