| Required:: ETL/Python Developer :: Location: Remote at Remote, Remote, USA |
| Email: [email protected] |
| Title: ETL/Python Developer Location: Remote Visa:: Non H1B 10+ Years of exp Manager Notes: "The stack is very ETL heavy, maintaining orchestrator's jobs, and supported Redshift data warehouse infrastructure Technologies: Talend, Terraform, Python, Scripting, Docker, AWS: EC2, S3 Job Details: 1. ETL tools and technologies used in this project - Involves working with but not limited to: Talend (Talend Studio, Talend Remote Engine, Talend Management Console), Terraform, Python, scripting, Docker, AWS services like EC2, S3. 2. What tools are used for orchestrating the ETL processes - Custom built solutions involving Talend. 3. Redshift Spectrum or focusing on traditional Redshift clusters - Mainly traditional Redshift clusters. 4. What other AWS services or tools are integrated into the project - The infrastructure consists of multiple aspects of Talend (Talend Remote Engine, Talend Managament Console, Talend Studio), EC2 instances used to run Talend, Redshift and various repositories (Python and Terraform) for adding new sources and to manage and support the infrastructure. 5. Size of data that we are working with - 8400 GB 6. Are integrating data from multiple sources into the Redshift warehouse, or building a data pipeline for real-time analytics - Majority of it is building the Redshift data warehouse. 7. What does the day to day look like - Building and maintaining ETL pipelines. Also monitoring daily jobs and address the failures if any. Also work on optimizing the performance of Redshift infrastructure using Terraform. 8. Does the job include designing and building the ETL pipeline vs. maintaining existing ones - Both 9. I generally support the Data Analytics' team's. What does supporting Redshift infrastructure mean in this context - It includes managing the security aspects and fulfilling requests that come in from Cloud Engineering team. - Maintaining Talend infrastructure. - Also includes working with Terraform to maintain all the aspects of the Redshift cluster infrastructure using terraform templates. Best Regards, Ravi Kumar Lead Technical Recruiter Excel Hire Staffing, LLC [email protected] || www.excelhirestaffing.com Linkedin Link:: https://www.linkedin.com/in/ravi-kumar-08264b78/ Address: 5856 Owens Drive, Pleasanton, CA 94588 -- Keywords: sthree information technology California Required:: ETL/Python Developer :: Location: Remote [email protected] |
| [email protected] View all |
| Thu Nov 21 04:05:00 UTC 2024 |