| AWS Data Engineering Lead Remote EST CST at Remote, Remote, USA |
| Email: [email protected] |
| From: Ashi, Key Infotek [email protected] Reply to: [email protected] Experience: 10-12+ years in Data Engineering, 5+ years in AWS cloud environment Job Overview: We are looking for a seasoned AWS Data Engineering Lead to design, architect, and develop end-to-end data pipelines for real-time use cases in the HR IT function. This role requires a blend of leadership and technical skills to guide the team and work hands-on in building scalable, secure, and efficient data solutions using AWS Glue and other AWS services. Knowledge of HR-related data and familiarity with Snowflake is a plus. Key Responsibilities: - Lead and architect end-to-end data pipeline solutions using AWS Glue for real-time data ingestion and transformation. - Design and develop scalable data architectures and solutions leveraging AWS S3, AWS IAM, AWS RDS, and AWS Lambda. - Implement event-based and scheduled pipelines using AWS Step Functions and AWS Lambda. - Ensure data security and compliance with organizational standards and AWS best practices. - Optimize the performance of data pipelines using Spark. - Collaborate closely with HR stakeholders to understand data needs and translate them into technical solutions. - Mentor and guide the development team, ensuring high-quality and timely delivery. - Coordinate with other teams, such as DevOps and Security, to ensure smooth implementation of solutions. - Maintain and monitor VPC subnets for secure data transfers. - Drive best practices for AWS architecture, Spark optimization, and cloud-native data engineering. Must-Have Skills (Need to be in resume and skill matrix): - AWS Glue: Strong experience in building ETL/ELT pipelines. - AWS S3: Experience with large-scale data storage solutions. - AWS IAM: Expertise in managing access control and data security. - AWS Lambda: Event-driven architecture experience. - AWS RDS: Strong understanding of working with relational databases in AWS. - AWS Step Functions: Experience designing and managing workflows. - Spark: Hands-on experience with Spark for data processing. - Event-Based and Scheduled Pipelines: Proven experience in managing event-driven architectures. - Basic Networking Knowledge: Understanding of VPC subnets and security considerations in AWS. Nice-to-Have Skills: - HR Data Knowledge: Familiarity with HR data attributes and functions. - Snowflake: Basic knowledge or experience working with Snowflake. - AWS CloudFormation/Terraform: Experience automating infrastructure provisioning. - CI/CD Pipelines: Experience with continuous integration and continuous deployment (CI/CD) tools like Jenkins or AWS CodePipeline. - Data Governance and Security: Understanding of data privacy regulations and governance in a cloud environment. Qualifications: - Bachelors degree in Computer Science, Information Technology, or related field. - AWS Certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data Specialty) are highly desirable. Keywords: continuous integration continuous deployment sthree information technology AWS Data Engineering Lead Remote EST CST [email protected] |
| [email protected] View all |
| Thu Sep 26 22:49:00 UTC 2024 |