| Senior Data Engineer Architect at Remote, Remote, USA |
| Email: [email protected] |
| From: Sonali Tiwari, Tek Inspirations [email protected] Reply to: [email protected] Job Description - Requirment of Senior Data Engineer Architect Hybrid in Boston. MA or Peachtree City, GA ( Local Highly Preferred) Please note that this is a senior level consultant with 15+ years of experience and It would be great to have Auto and Manufacturing experience. Lots of Data Modeling, Data warehouse, and data lake experience needed. Tech and platform exp are not important. Data structure, Big data systems + advantage of data structure Highly skilled in SQL & Python data expiration **very Impt Must have** Domain knowledge of Automotive, Manufacturing, Supply Chain & Inventory SAP backend exp Intake call Notes: This person will grow into a Data Architect role and will be the backbone of the entire team Must have conceptual experience; not just the background Should be able to explain clearly how they came up with the solution The manager comes with 25yrs of Data space and will ask detailed questions during the interview process. Sr. Data Engineer (PASA) #967505 Job Description: Senior Data Engineer is responsible for building and maintaining the data infrastructure that enables an organization to efficiently collect, process, store, and analyze data for informed decision-making. Their role is critical in ensuring data availability, quality, and reliability throughout the data lifecycle. Accountabilities: Data Pipeline Architecture Design: responsible for designing and architecting data pipelines that move, transform, and process data from various sources to target data storage or analytics platforms and need to ensure scalability, reliability, and efficiency in these architectures. Data Modeling: They design and implement data models that suit the requirements of the project and structuring data to support efficient querying and analysis. ETL (Extract, Transform, Load) Processes: Building robust ETL processes is a key responsibility. Senior Data Engineers design and develop ETL workflows that extract data from source systems, transform it into the desired format, and load it into Azure Data Lake. Data Integration: integrating data from different sources, which may include databases, APIs, third-party services, and more. Integration involves handling data in various formats and ensuring data consistency and accuracy. Data Quality and Monitoring: implement data quality checks and monitoring mechanisms to identify and rectify data quality issues. Also create alerts and notifications for anomalies in data processing. Performance Optimization: responsible for optimizing the performance of data pipelines and processing systems. This includes fine-tuning queries, optimizing data storage, and managing resources effectively to ensure timely data processing. Data Security: Ensuring data security and implement security measures to protect sensitive data and ensure proper access controls. Collaboration: collaborate with cross-functional teams, including data analysts, business analysts and software engineers, to understand data requirements and provide the necessary guidelines to support their work. Technology Selection: assess and select appropriate tools, technologies, and frameworks for different aspects of the data engineering process and stay up to date with industry trends and emerging technologies. Leadership and Mentoring: As senior members of the team, should be able take on leadership roles by guiding and mentoring junior data engineers. Participate in code reviews, best practice discussions, and training sessions. Troubleshooting and Support: When issues arise in data pipelines or systems, Senior Data Engineer is responsible for diagnosing and resolving these issues in a timely manner to minimize disruptions. Documentation: document data pipelines, workflows, architecture designs, and other relevant processes to ensure knowledge sharing and maintainable systems. Scope and Competency Requirements: Problem-Solving/Know How: (complexity of problems and education/experience & other knowledge required) A bachelors or masters degree in computer science, information technology, data science, or a related field with 7+ years of experience as a data engineer, with a significant portion of that experience specifically working with Azure data services. Proficiency in Microsoft Azure services and tools related to data engineering, such as Azure Data Factory, Azure Databricks, Azure Analytical Service, Azure Synapse Analytics, Azure Cosmos DB, etc. Strong Experience in implementing data solutions in Azure Data Lake platform Experience with Data Governance tools like Azure Purview, Alation.etc are preferred. Expertise in SQL, T-SQL and experience working with various database systems, including relational databases and NoSQL databases is required. Experience with data modeling, schema design, and performance optimization, especially creating tabular models for Self-service analytics using Power BI. Strong skills in data integration, ETL (Extract, Transform, Load) processes, and data orchestration workflows. Expertise in data warehousing concepts and techniques. Programming skills in languages such as Python, PySpark, Java, Scala, or PowerShell for scripting and automation. Experience designing, building, and optimizing end-to-end data pipelines. Proven track record of implementing data solutions that meet performance, scalability, and reliability requirements. Experience with data migration, data warehousing, and data lake solutions. Experience in creating data pipelines to extract the data from SAP applications is preferred. The ability to analyze complex data engineering challenges and develop innovative solutions. Strong troubleshooting skills to identify and resolve issues in data pipelines and systems. Effective communication skills to work collaboratively with cross-functional teams, including data scientists, analysts, and business stakeholders. The ability to translate technical concepts into understandable terms for non-technical team members. Data accuracy and quality are paramount. A Senior Azure Data Engineer should be meticulous in ensuring that data pipelines and processes maintain high data integrity. As a senior role, this individual might be required to lead and mentor junior members of the data engineering team. ATTRIBUTES/SKILLS: Functional knowledge and ability of Systems and Software. Must be a capable team player and able to adapt to various project requirements. Must be able to use methods appropriate to the position and assignment to solve problems and achieve accurate results. Excellent organizational skills for multi-tasking. Demonstrated ability to work in a cross-functional, team oriented environment. Must have the ability to remain organized and self directed in a fast-paced environment. Must have ability to analyze, interpret, and communicate related data, including analytical data. Must have the ability to develop solutions for problems or systems. Must be able to work with minimal supervision displaying strong self-motivation. Must have the ability to learn computer applications as necessary for the position. Must successfully complete any training or certification courses required to support or strengthen position. Must have the ability to read, interpret, and follow verbal and written instructions, work procedures and other related materials. COMMUNICATIONS: Persuasive, professional communication style. Must possess strong verbal and written communication skills including strong customer service skills. Must communicate appropriate information accurately and clearly as needed including strong presentation skills. Must have the ability to effectively and appropriately interface and communicate with a variety of individuals or groups on a frequent basis. MANAGEMENT: ($, employees, and other resources managed. Attach organization chart) NA TRAVEL: Ability to travel as required for the position. Quality Procedures: Review procedures, materials, or processes to assure compliance to all specifications and standards. Comply with and enforce all procedures and work instructions. Respond to quality and work issues and take corrective action as appropriate. Has the organizational authority to identify and segregate, as necessary, suspect or non-conforming material (for example components, saleable product, specifications, controlled documents, etc.) and to notify the appropriate authority (manager, etc.) for instructions. Safety and EPA Compliance: Maintains tools, equipment, and work areas in a neat and orderly manner and observes all work-related safety practices and procedures. Will comply with all applicable Company safety policies, rules, and regulations. Must comply with all Federal, State, and local environmental laws and regulations and relevant company procedures and instructions related to the environmental management system and contribute to the achievement of the relevant environmental objectives and targets established by the Company. In addition, provides the leadership and resources for such activities as applicable. Physical Demands Required to sit, stand, or walk for prolonged periods of time. Use hands and fingers to handle or feel objects or use controls, tools, or equipment. May require the ability to use a keyboard and other office equipment. May require specific vision abilities that include close vision. May be required to perform concentrated work for extended periods of time. May be required to lift, carry, or exert items or objects of reasonable weight. May be required to perform duties beyond standard business hours. Must have the ability to perform the essential functions of the job with or without reasonable accommodation. This above description covers the most significant key result areas but does not exclude the assignment of other occasional responsibilities and accountabilities. Keywords: business intelligence database information technology Georgia Massachusetts |
| [email protected] View all |
| Wed Oct 25 18:08:00 UTC 2023 |