| Data Pipeline Engineer (ETL Glue) developer- 100% remote at Remote, Remote, USA |
| Email: [email protected] |
| From: pankaj, Stellent IT [email protected] Reply to: [email protected] Data Pipeline Engineer (ETL/Glue) developer 100% remote Phone and Skype Long Term Job Description: Glue Pyspark Python CI/CD not just awareness, real experience This particular team treats this like a systems engineer / software engineer role where they develop for reusability by other functional organizations. In which roles have candidates used modern software delivery methods like TDD, BDD, CI/CD Our client is looking to hire Data Pipeline Engineer (ETL/Glue) developer to build and deploy modular data pipeline components such as Apache Airflow DAGs, AWS Glue jobs, AWS Glue crawlers through a CI/CD process. You will join a growing Data Engineering team and participate in design and build of data ingestion and transformation pipelines based on the specific needs driven by Product Owners and Analytics consumers. You must possess strong knowledge and interest in data processing and have a background in data engineering. Candidate will work directly with senior data engineers, solution architects, DevOps engineers, product owners and data consumers to deliver data products in a collaborative and agile environment. You will also have to continuously integrate and push code into cloud production environments. To be considered, you must have Experience in AWS Data processing, Analytics, and storage Services such as Simple Storage Service (s3), Glue, Athena and Lake Formation Experience in extracting and delivering data from various databases such as MongoDB, DynamoDB, SnowFlake, Redshift, Postgres, RDS Coding experience with Python, SQL, yaml, spark programming (pyspark) Hands on experience with Apache Airflow as a pipeline orchestration tool Experience in AWS Serverless services such as Fargate, SNS, SQS, Lambda Experience in Containerized Workloads and using cloud services such as AWS ECS, ECR and Fargate to scale and organize these workloads. Experience in data modeling and working with analytics teams to design efficient data structures. Applied knowledge of working in agile, scrum, or DevOps environments and teams Applied knowledge of modern software delivery methods like TDD, BDD, CI/CD Applied knowledge of Infrastructure as Code (IAC) Experience with development lifecycle (development, testing, documentation, and versioning) Preferred skills/certifications (AWS Certified Developer Associate, AWS Certified Big Data Specialty, Gitlab CI/CD) Keywords: continuous integration continuous deployment sthree information technology |
| [email protected] View all |
| Tue Sep 19 21:09:00 UTC 2023 |