| Data Engineer :: Remote :: GC-EAD or H4 or USC only:: Max Rate is $60 per hr on C2C :: 11+ Years not below at Max, North Dakota, USA |
| Email: [email protected] |
| From: Mohd Niyaz, Sibitalent [email protected] Reply to: [email protected] Data Engineer Overview: Seeking a skilled Data Engineer to join a dynamic team responsible for building and maintaining a Data Lake. The team leverages AWS Redshift to consume and store data from various sources, making it available for both internal and external stakeholders. The core of the work revolves around establishing seamless system-to-system connections via ETL processes and transforming data files for integration into the Data Lake. The primary database is Redshift, supported by DynamoDB for configuration management and EMR as the processing framework. Additionally, Postgres is utilized as the database type, and Python scripts along with Lambda functions are key tools for data ingestion and transformation. The team operates in an Agile environment, following Scrum methodology with 2-week sprints. This role requires a comprehensive understanding of the full stack of technologies used, as team members are expected to contribute across all areas of the data management ecosystem. Responsibilities: Develop and maintain ETL processes to load data into the Data Lake using Python, Lambda functions, and other AWS services Design and implement data ingestion pipelines that handle various file types and formats, ensuring they are parsed and prepared for consumption Work with AWS Redshift to manage data storage and retrieval, ensuring optimal performance and data integrity Utilize EMR for data processing tasks and DynamoDB for configuration management Collaborate with team members in a full-stack environment, contributing to all aspects of data engineering from ingestion to storage Participate in Agile ceremonies, contributing to sprint planning, daily stand-ups, and retrospective meetings Required Skills: 4+ years of experience in data engineering 4+ years of experience working with AWS Redshift and EMR Strong experience with AWS Lambda and Python for building data pipelines Experience with Postgres databases Knowledge of DynamoDB for configuration management Strong understanding of ETL processes and data lake architecture Familiarity with Agile/Scrum methodologies, particularly in a JIRA-managed environment. Thanks and Regards, Mohd Niyaz Email : [email protected] Linkedin ID:- linkedin.com/in/mohd-niyaz-362667220 Contact : +1 (936)-310-2126 Web: www.sibitalent.com 101 E. Park Blvd., Suite 600 Plano, TX - 75074 Keywords: information technology Idaho Texas Data Engineer :: Remote :: GC-EAD or H4 or USC only:: Max Rate is $60 per hr on C2C :: 11+ Years not below [email protected] |
| [email protected] View all |
| Tue Aug 13 22:01:00 UTC 2024 |