| Lead Data Engineer || Phoenix, AZ (Local Only) at Phoenix, Arizona, USA |
| Email: [email protected] |
| Role: Lead Data Engineer Location: Phoenix, AZ (Local Only) Duration: Long Term Interview: Video Job Description: 10+ years of experience Must Have: Informatica, Python, Spark, AWS Services with EMR, Iceberg, GIT, Airflow, DBT, Trino, Snowflake, DBT, Linux/UNIX, Airflow Lead and mentor a team of data engineers, fostering a collaborative and innovative environment. Define best practices and standards for data engineering processes and methodologies. Data Pipeline Development: Design, develop, and maintain ETL processes using Informatica and Python to ensure efficient data movement and transformation. Implement and manage workflows in Apache Airflow for scheduling and monitoring data pipelines. Data Architecture: Collaborate with data architects and analysts to develop scalable and efficient data models. Optimize existing data pipelines for performance, reliability, and cost-effectiveness. Data Quality and Governance: Establish data quality metrics and monitoring processes to ensure the integrity and accuracy of data. Implement data governance practices to comply with regulatory requirements and internal policies. Collaboration: Work closely with cross-functional teams including data scientists, analysts, and business stakeholders to understand data needs and deliver solutions that drive business value. Participate in the planning and execution of data-related projects. Documentation and Reporting: Maintain comprehensive documentation of data engineering processes, architectures, and workflows Provide regular updates to stakeholders on project status, challenges, and solutions. Experience: 10+ years of experience in data engineering or a related field. 5+ years of experience leading data engineering teams. Technical Skills: Strong expertise in Informatica for ETL processes. Proficient in Python for data manipulation and automation. Experience with Apache Airflow for workflow management. Familiarity with cloud platforms (AWS) and big data technologies (Hadoop, Spark) is a plus. Solid understanding of SQL and database management systems (e.g., MySQL, PostgreSQL, NoSQL). Thanks Arun Solani [email protected] -- Keywords: information technology Arizona Lead Data Engineer || Phoenix, AZ (Local Only) [email protected] |
| [email protected] View all |
| Wed Oct 23 00:23:00 UTC 2024 |