| need local :Lead Data Engineer || Illinois, IL (Hybrid) || long term at Remote, Remote, USA |
| Email: [email protected] |
| From: Karthik, Hclglobal [email protected] Reply to: [email protected] Job Title-Lead Data Engineer Location -Illinois, IL (Hybrid) Mandotory Skill- Lead Mong DB Azure Need Only : H1B Visa ,H4EAD, USC Need Only Local Illinois, IL (Hybrid) Mandatory Skills Data Engineering, Logic Apps, Azure Service Bus, SQL, Azure data factory, Mongo db "Overview: The Lead Data Engineer will be responsible for the overall design and development of the solution; works with other data Engineers and modelers to implement various processes to support building necessary components and maintaining the platform. Lead data engineer interfaces with business teams to gather high level requirements. Responsibilities: Design, model, and implement data warehousing activities to deliver the data foundation that drives impact through informed decision making. Work with engineers, product managers, and data analysts to understand data needs, propose or enhance existing framework and platform. Lead planning and designing solutions for multiple project initiatives, provide technical management to other Data Engineers Handle Client communications and settle expectations. Solve challenging data integration problems utilizing optimal ETL, ELT patterns, frameworks, query techniques, and sourcing from structured and unstructured data sources. Leverage MongoDB as a NoSQL database solution, managing data modelling, indexing, and query optimization. Develop Azure Logic Apps to automate business workflows, integrating with various external systems through connectors and custom APIs. Design and implement Azure Service Bus solutions for enterprise-scale applications, enabling seamless communication between microservices and distributed components. Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts. Build data expertise and own data quality for allocated areas of ownership including managing SLAs and supporting processes running in production. Required Skills & Qualifications: A degree (preferably an advanced degree) in Computer Science, Engineering or a related field Capable in planning, designing and leading small team of data Engineers including mentoring and guiding development. 7+ years of industry experience in architecting and developing data warehousing solutions, preferably in large production environments. Azure Data Service - ADF or Synapse Analytics Extensive experience in designing, developing, and maintaining data solutions/pipelines in the Azure ecosystem, including Azure Service Bus, Logic Apps, SQL, Mongo DB & ADF. Expertise in data ETL/ELT, schema design, dimensional modeling, slowly changing dimensions. Capability in coding and optimizing PySpark programs, partitioning and scaling parquet, and delta formatted data structures. Creating and optimizing large scale datasets, and familiarity with modern data warehousing techniques Keywords: database Illinois |
| [email protected] View all |
| Wed Jan 31 02:49:00 UTC 2024 |