| Job id (748674) Senior Big Data Engineer (Remote) at Bismarck, North Dakota, USA |
| Email: [email protected] |
| From: Lokesh, GSK SOLUTIONS INC [email protected] Reply to: [email protected] Job Title: Senior Big Data Engineer (Remote) Location: Bismarck, ND Duration:6+ Months Job Type: Corp-Corp / 1099 / Contract W2 Client: State of North Dakota Interview Type: Webcam Note: This is a 35 hours/week schedule and Candidate MUST work during CST timings Job Description: Develops, engineers, maintains, runs tests, evaluates, and implements big data infrastructure, projects, tools, and solutions, working with the latest database technologies, to get results from vast amounts of data quickly. The Big Data Engineer is a vital member of a collaborative team, responsible for designing, engineering, maintaining, testing, evaluating, and implementing big data infrastructure, tools, projects, and solutions for the North Dakota University System (NDUS).This role involves working closely with the team to leverage cutting-edge database technologies for the swift retrieval of results from vast datasets. The engineer will select and integrate big data frameworks and tools to meet specific needs and manage the entire lifecycle of large datasets to extract valuable insights. Key Responsibilities: Design and implement scalable big data solutions tailored to NDUSs needs. Maintain and enhance existing big data infrastructures to meet NDUSs unique requirements. Test and evaluate new big data technologies and frameworks for compatibility with NDUS systems and goals. Collect, store, process, manage, analyze, and visualize large datasets to derive actionable insights. Collaborate with team members to integrate big data solutions with existing NDUS systems. Ensure data integrity and security across all platforms used within NDUS. Develop and optimize data pipelines for ETL/ELT processes specific to NDUSs data needs. Document technical solutions and maintain comprehensive records in line with NDUS standards and protocols. Stay updated with the latest trends and advancements in big data technology relevant to NDUSs strategic initiatives. Required Qualifications: Thorough understanding of cloud computing technologies, including IaaS, PaaS, and SaaS implementations. Skilled in exploratory data analysis (EDA) to support ETL/ELT processes. Proficiency with Microsoft cloud products, including Azure and Fabric. Experience with tools such as Data Factory and Databricks. Ability to script in multiple languages, with a strong emphasis on Python and SQL. Preferred Qualifications: Experience with data visualization tools. Proficiency with Excel and Power BI. Familiarity with Delta Lake. Knowledge of Lakehouse Medallion Architecture. Required/Desired Skills Skill Required /Desired Amount of Experience IaaS, PaaS, or SaaS data or AI implementations within Microsoft Azure Required 3 Years Exploratory Data Analysis (EDA): Proficiency in EDA techniques to support ETL/ELT processes Required 3 Years Implementation of Development and Production Workflows in Azure Data Factory and Databricks Required 2 Years Strong scripting skills in Python and SQL Required 3 Years Expert proficiency of data engineering creation in Microsoft Fabric Required 1 Years Expert proficiency with Excel and Power BI Highly desired 2 Years Expert proficiency of Delta Lake format and protocol Highly desired 1 Years Expert understanding of the Data Lakehouse Medallion Architecture Highly desired 1 Years Lokesh Talent Acquisition Specialist GSK Solutions Inc | An E-Verify Company Lokeshgsksolutions.com 832-990-2426 linkedin.com/in/bupathi-lokesh-victor 633 E Fernhurst Drive, Unit 1002, Katy, TX 77450 Keywords: artificial intelligence business intelligence wtwo North Dakota Texas Job id (748674) Senior Big Data Engineer (Remote) [email protected] |
| [email protected] View all |
| Sat Nov 09 01:52:00 UTC 2024 |