| Airflow GCP Engineer at Sunnyvale, California, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1216970&uid= From: Utsav (IT Resource Manager), ChabezTech [email protected] Reply to: [email protected] **Job Title:** Senior Airflow GCP Engineer **Location:** Sunnyvale, CA (Onsite) **Duration:** Long Term **Responsibilities:** 1. **Design and develop data pipelines:** Responsible for architecting, designing, and implementing scalable data pipelines and big data applications on Google Cloud Platform (GCP) utilizing Apache Airflow, Spark, and other relevant technologies. The focus will be on efficiently handling large volumes of data. 2. **Complex SQL query development:** Develop and optimize complex SQL queries for data extraction, transformation, and loading (ETL) processes. Ensure data integrity, performance optimization, and adherence to best practices in SQL development. 3. **Workflow automation with Apache Airflow:** Lead the design, implementation, and management of data processing workflows and automation using Apache Airflow. This involves designing, scheduling, and orchestrating tasks for smooth and reliable execution of data pipelines. 4. **Utilization of Dataproc and BigQuery:** Utilize Google Cloud Dataproc for large-scale data processing tasks, leveraging Apache Spark and other distributed computing frameworks. Additionally, design, optimize, and maintain data schemas and structures on BigQuery for efficient data storage and retrieval. 5. **Support and maintenance:** Provide ongoing support and maintenance for existing data systems and pipelines, including troubleshooting issues, optimizing performance, and implementing enhancements. Participation in rotational on-call support may be required. 6. **Collaboration and communication:** Collaborate closely with cross-functional teams to understand business requirements and translate them into technical solutions. Effective communication skills are essential for sharing knowledge, presenting solutions, and aligning with project objectives. Thanks & Regards Utsav Manager ChabezTech LLC 4 Lemoyne Dr #102, Lemoyne, PA 17043, USA Email: [email protected] | www.chabeztech.com - **Education:** Bachelor's degree in computer science or a related field is preferred. Equivalent experience may be considered. **Preferred Experience:** - Gitflow. - Atlassian products BitBucket, JIRA, Confluence, etc. This role offers an exciting opportunity to work with cutting-edge technologies in a dynamic environment. If you are passionate about GCP data engineering and possess the required skills and experience, we encourage you to apply now! Keywords: information technology California Pennsylvania http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=1216970&uid= |
| [email protected] View All |
| 08:10 PM 14-Mar-24 |