| Data Engineer at Remote, Remote, USA |
| Email: [email protected] |
| From: Santhoshi, HAN IT Staffing [email protected] Reply to: [email protected] Role : Data Engineer - Onsite (Need Local) Work location: HARRISBURG (US:17105), PA Client : Capgemini Job Description : .As an AWS Redshift Data Engineer, the primary responsibility is to design, implement, and maintain data solutions using Amazon Redshift. Your role involves working with large-scale data sets, optimizing performance, and ensuring the reliability and availability of the Redshift environment. Here is a general job description for an AWS Redshift Data Engineer: 1. Data Modeling and Design: Develop and maintain data models for Redshift databases, including schema design, table structures, and optimization techniques. Collaborate with data architects and stakeholders to understand requirements and translate them into efficient data structures. 2. ETL Development: Design and implement Extract, Transform, Load (ETL) processes to extract data from various sources, transform it as per business requirements, and load it into Redshift. Develop efficient and scalable ETL workflows, considering data quality, performance, and data governance. 3. Performance Optimization: Optimize query performance by creating appropriate data distribution keys, sort keys, and compression techniques. Identify and resolve performance bottlenecks, fine-tuning queries, and optimizing data processing to enhance Redshift's performance. 4. Data Integration: Integrate Redshift with other AWS services, such as AWS Glue, AWS Lambda, Amazon S3, and more, to build end-to-end data pipelines. Ensure seamless data flow between different systems and platforms, maintaining data integrity and consistency. 5. Monitoring and Troubleshooting: Implement monitoring and alerting systems to proactively identify issues and ensure the stability and availability of the Redshift cluster. Perform troubleshooting, diagnose and resolve data-related issues, and work closely with support teams to resolve any performance or operational problems. 6. Security and Compliance: Implement security best practices to protect data stored in Redshift. Ensure compliance with data privacy regulations and industry standards, such as GDPR and HIPAA. Implement encryption, access controls, and data masking techniques to secure sensitive data. 7. Documentation and Collaboration: Maintain documentation of data models, ETL processes, and system configurations. Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and provide data solutions that meet their needs. 8. Scalability and Capacity Planning: Plan and execute strategies for scaling Redshift clusters to handle increasing data volumes and user demands. Monitor resource utilization, track data growth, and make recommendations for capacity planning and infrastructure scaling. 9. Knowledge or previous experience in oracle PLSQL will be added advantage. Keywords: sthree information technology Pennsylvania Data Engineer [email protected] |
| [email protected] View all |
| Tue Jul 23 18:56:00 UTC 2024 |