Email Alert for
ⓘ There was an unexpected error processing your request.
Please refresh the page and try again.
If the problem persists, please contact us with your issue.
You can always manage your preferences and update your interests to ensure you receive the most relevant opportunities.
Get ready to discover your next great opportunity.
Saxon Global
Jersey City, NJ, United States
The Role: You'll work with a cross-functional team of research scientists and engineers to contribute to our software and data platforms. Drive the design and implementation of data models, ETL pipelines, and data processing logic while working closely with the other teams in order to translate operational data with microservices into business valu
Job Source: Saxon GlobalCiti
Jersey City, NJ, United States
About Citi Citi, the leading global bank, has approximately 200 million customer accounts and does business in more than 160 countries and jurisdictions. Our core activities are safeguarding assets, lending money, making payments and accessing the capital markets on behalf of our clients. Citi’s Mission and Value Proposition explain what we do and
Job Source: CitiThe Cypress Group
New York, NY, United States
My client is a profitable, growing, software company that builds the leading ad solution in the country. Their services consume huge volumes of data from a variety of sources. They utilize a home-grown Hadoop-based big data analytics platform to execute analytics and data mining on the data sets. They are currently looking for a talented Software E
Job Source: The Cypress GroupAbode Techzone LLC
Jersey City, NJ, United States
Job Description: Looking for consultants local to NY, NJ & CT Mainly we are looking for AWS Not Azure OR GCP NEED EXTENSIVE EXPERIENCE WORKING WITH BANKING CLIENTS Looking for 9 Plus years of experience As a Software Engineer within the Corporate Investment Banking Treasury Services, you are part of an agile team that works to enhance, design
Job Source: Abode Techzone LLCS&P Global
New York, NY, United States
About the Role: Grade Level (for internal use): 12 About the Role: Title: Lead Data Engineer – Big Data The Team: You will be an expert contributor and part of the Rating Organization’s Ingestion Pipelines Engineering Team. This team, who has a broad and expert knowledge on Ratings organization’s critical data domains, technology stacks and a
Job Source: S&P GlobalIntegral Ad Science
New York, NY, United States
Integral Ad Science (IAS) is a global technology and data company that builds verification, optimization, and analytics solutions for the advertising industry and we’re looking for A Senior Data Engineer to join our team. If you are excited by technology that has the power to handle hundreds of thousands of transactions per second; collect tens o
Job Source: Integral Ad ScienceYahoo
Little Ferry, NJ, United States
Yahoo Mail is the ultimate consumer inbox with hundreds of millions of users. It’s the best way to access your email and stay organized from a computer, phone or tablet. With its beautiful design and lightning fast speed, Yahoo Mail makes reading, organizing, and sending emails easier than ever. A Little About Us The Yahoo Mail engineering team d
Job Source: YahooHudson River Trading
New York, NY, United States
Hudson River Trading (HRT) is looking for Systems Engineers to join our growing Research & Development team. This team builds and maintains an exceptionally large and growing distributed compute cluster, many-PB-scale storage systems, operating systems, automation software, and development tools. We are seeking a highly skilled engineer with exper
Job Source: Hudson River TradingNew York, NY, United States
Big Data Engineer
Hybrid-Atlanta GA
12+ Months Contract
Phone and Skype
Visa: OPT/H4-EAD/GC-EAD/GC/USC
Job Description
Client Corporation is currently seeking an experienced Data Engineer - Big Data individual for their Midtown office in Atlanta, GA. The successful candidate must have Big Data engineering experience and must demonstrate an affinity for working with others to create successful solutions. Join a smart, highly skilled team with a passion for technology, where you will work on our state of the art Big Data Platforms. They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements. The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties.
Skills and Experience (years depend on seniority level):
Required:
•6+ years of overall IT experience
•3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming
•Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
•Deep knowledge of troubleshooting and tuning Spark applications
•3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
•3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or
•3+ years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets
•2+ years of experience with Python (and/or Scala) and PySpark/Scala-Spark
•3+ years of experience with Cloud platforms e.g. AWS, GCP, etc.
•3+ years of experience with database solutions like Kudu/Impala, or Delta Lake or Snowflake or BigQuery
•2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
•Experience in successfully building and deploying a new data platform on Azure/ AWS
•Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue
•Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series
•Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog
•Knowledge of Unix/Linux platform and shell scripting is a must
•Strong analytical and problem-solving skills
Preferred (Not Required):
•Strong SQL skills with ability to write intermediate complexity queries
•Strong understanding of Relational & Dimensional modeling
•Experience with GIT code versioning software
•Experience with REST API and Web Services
•Good business analyst and requirements gathering/writing skills
Email Alert for Big Data Engineer jobs in New York, NY, United States
ⓘ There was an unexpected error processing your request.
Please refresh the page and try again.
If the problem persists, please contact us with your issue.
You can always manage your preferences and update your interests to ensure you receive the most relevant opportunities.
Get ready to discover your next great opportunity.