| AWS Data Engineer: Remote at Remote, Remote, USA |
| Email: [email protected] |
| Hi, I hope you are doing great. Kindly find the below-mentioned job description and if interested, share your updated resume at [email protected] Role: Data Engineer (AWS) Location: REMOTE Must live in the United States (25672) Duration: Long Term Interview: Video Visa: No H1B PLEASE NOTE YEARS OF EXPERIENCE REQUIRED Duties: Designing, implementing, and maintaining batch and streaming data pipelines between various SQL sources, including Aurora and SQL Server, and a target data lake in S3, as well as enterprise data stored in Redshift/Snowflake. Expertise in AWS cloud services, such as Kinesis, S3, Lake Formation, Glue, and Step Functions, to build scalable, reliable, and high-performance data pipelines that enable seamless data integration and empower data-driven insights. Data Pipeline Design: Design end-to-end data pipelines that efficiently extract, transform, and load data from SQL sources (Aurora, SQL Server) to the target data lake in S3 and the enterprise data in Redshift/Snowflake. Batch and Streaming Integration: Implement both batch and real-time streaming data integration solutions using AWS Kinesis and other relevant technologies. Data Transformation: Develop data transformation processes using AWS Glue or other ETL tools to harmonize, cleanse, and enrich data for analytical use. Data Lake Management: Oversee the setup and configuration of the data lake in S3, applying AWS Lake Formation best practices for data organization, cataloging, and access control. Data Governance: Ensure adherence to data governance and security standards across the data pipelines, guaranteeing data privacy and compliance. Performance Optimization: Continuously monitor and optimize the performance of the data pipelines, addressing bottlenecks and ensuring efficient data processing and delivery. Error Handling and Monitoring: Implement error handling mechanisms and robust data monitoring to identify and resolve data pipeline issues proactively. Data Cataloging and Lineage: Establish and maintain data cataloging and lineage information using AWS Glue Data Catalog to enable data discoverability and traceability. Documentation: Create comprehensive technical documentation, including design specifications, data flow diagrams, and operational guides. Collaboration: Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and deliver reliable data solutions. Data Governance: Ensure data governance principles are implemented throughout the data pipelines to maintain data quality and integrity. Education Qualification: Bachelor's Degree from an accredited college or university with a major in Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline. General Experience: At least six (6) years of experience working on AWS cloud-based batch and streaming data pipelines. Strong proficiency in AWS cloud services, including Kinesis, S3, Lake Formation, Glue, and Step Functions. In-depth knowledge of SQL databases, such as Aurora and SQL Server, and data lakes in S3, as well as enterprise data in Redshift/Snowflake. Hands-on experience with ETL tools, data transformation, and data integration techniques. Familiarity with data governance, data privacy, and security best practices in AWS environments. Strong problem-solving skills and the ability to troubleshoot complex data pipeline issues. Excellent communication and teamwork skills to collaborate effectively with cross-functional teams. AWS certifications, such as AWS Certified Data Analytics - Specialty or AWS Certified Big Data - Specialty, are advantageous. Thanks & Regards Harshit Sharma | 1Point System LLC Technical Recruiter Office: (704) 980-0370 Fax: 803-832-7973 [email protected] LinkedIn: linkedin.com/in/harshit-sharma-032388212 115 Stone Village Drive Suite C Fort Mill, SC 29708 An E-Verified company | An Equal Opportunity Employer -- Keywords: sthree information technology South Carolina |
| [email protected] View all |
| Tue Sep 05 21:51:00 UTC 2023 |