Data Engineer

Data EngineerWe need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National Guard and the Department of War (DoW). You will support operations on the Minuteman project team, which manages the NGB's enterprise data environment within the DoW's central analytics platform, and plays a crucial role in designing, developing, and maintaining the Minuteman Community Space's data infrastructure and systems. In this role, you will design and implement end-to-end data pipelines using ETL processes and technologies including Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, working closely with cross-functional teams to understand data requirements and design optimal data models and architectures that ensure efficient data processing and analysis. This is a full-time opportunity. We can offer a competitive salary, and a comprehensive benefits package.Responsibilities:Design, develop, and implement end-to-end data pipelines, utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQLWork closely with cross-functional teams to understand data requirements and design optimal data models and architecturesCreate and optimize data pipelines from scratch, ensuring scalability, reliability, and high-performance processingPerform data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasetsLeverage big data technologies to efficiently process and analyze large datasetsCollaborate with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision-making processesTroubleshoot data-related problems and provide innovative solutions to address complex data challengesImplement and enforce data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practicesResearch, design, and develop software and specialized utility programs in support of mission requirementsAnalyze user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysisUpdate and enhance existing software capabilitiesDevelop specifications and performance requirements, coordinating with engineering counterparts to integrate hardware and software systems as requiredMaintain databases within the application area, working individually or coordinating database development as part of a teamRequired Qualifications:Three (3) to ten (10) years of directly related experienceAvailable to work onsite in Arlington, VA 1-2 days per week3+ years of experience as a Data EngineerProficiency in ETL processes and firsthand experience with technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter NotebooksExperience working with large datasets and leveraging big data technologies to process and analyze data efficientlyUnderstanding of data modeling/visualization, database design principles, and data governance practicesPreferred Qualifications:Experience with the Department of War (DoW) mission and associated datasetsPrevious experience with Advana or the War Data PlatformExperience with APIsExperience with Machine Learning models and/or Generative AIExperience building within Palantir Foundry applications, including Workshop, Ontology Manager, and GaiaEducation:B.A./B.S. or M.A./M.S. in a related fieldLocation:National Capital Region. Onsite in Arlington, VA 1-2 days per week – RequiredClearance:Active Secret security clearanceU.S. citizenship is required since this role supports the U.S. Federal governmentClient:OUSD Personnel & Readiness (P&R), Minuteman project teamWork Hours:40 hours per week. 8 hours per dayEmployment Classification:Employment Classification Eligibility — W2Compensation:Salary range: $155,000 - $165,000Benefits: Benefits package includes options for health, dental, and vision insurance coverage; 401k contribution optionsWest 4th Strategy is an Equal Opportunity (EEO) employer. All qualified applicants will receive consideration for employment without regard to race, color, creed, religion, gender, sexual orientation, ancestry, national origin, age, marital status, mental disability, physical disability, medical condition, pregnancy, political affiliation, military or veteran status, or any other basis prohibited by federal or state law. Other Considerations: applicants will be subject to a background investigation. Individual's primary workstation is located in an office area. The noise level in this environment is low to moderate. Regularly required to sit for extended periods up to 80% of the time; frequently required to move about to access file cabinets and use office equipment such as PC, copier, fax, telephone, cell phone, etc. Occasionally required to reach overhead, bend, and lift objects of up to 10 lbs. Specific vision abilities required by this job include the use of computer monitor screens up to 80% of the time.