| Data Engineer with snowflake and Change Data Capture(CDC) Exp. -C2C-Chicago-IL Hybrid - C2C at Snowflake, Arizona, USA |
| Email: [email protected] |
| From: balamurgan, vysystems [email protected] Reply to: [email protected] Key Responsibilities:(12 - 14 years of Experience) Data Pipeline Development: Design, build, and manage data pipelines for the ETL process, using Airflow for orchestration and Python for scripting, to transform raw data into a format suitable for our new Snowflake data model. Data Integration: Implement and maintain data synchronization between on-premises Oracle databases and Snowflake using CDC tools. Support Data Modeling: Assist in developing and optimizing the data model for Snowflake, ensuring it supports our analytics and reporting requirements. Reporting Support: Collaborate with data architect to ensure the data within Snowflake is structured in a way that supports efficient and insightful reporting. Technical Documentation: Create and maintain comprehensive documentation of data pipelines, ETL processes, and data models to ensure best practices are followed and knowledge is shared within the team. Qualifications Experience: Total 10+ years of experience as data engineer; proven experience as a Snowflake data engineer for 5+ years. Skills: Strong knowledge of SQL writing complex queries, performance tuning, Strong experience with Oracle, Snowflake, ETL/ELT tools Data engineering: proven track record of developing and maintaining data pipelines and data integration projects Orchestration Tools: Experience in Airflow for managing data pipeline workflows. Programming: proficiency in Python and SQL for data processing tasks. Data Modeling: Understanding of data modeling principles and experience with data warehousing solutions. Cloud Platforms: Knowledge of cloud infrastructure and services, preferably Azure, as it relates to Snowflake integration. Collaboration Tools: Experience with version control systems (like Git) and collaboration platforms. CI/CD Implementation: Utilize CI/CD tools to automate the deployment of data pipelines and infrastructure changes, ensuring high-quality data processing with minimal manual intervention. Communication: Excellent communication and teamwork skills, with a detail-oriented mindset. Strong analytical skills, with the ability to work independently and solve complex problems. Certifications: Snowflake certification is a plus. Keywords: continuous integration continuous deployment information technology Data Engineer with snowflake and Change Data Capture(CDC) Exp. -C2C-Chicago-IL Hybrid - C2C [email protected] |
| [email protected] View all |
| Thu Nov 07 05:16:00 UTC 2024 |