| Lead Data engineer - Chicago, IL- oniste at Chicago, Illinois, USA |
| Email: [email protected] |
| From: Vikas Rai, Samson Software Solution [email protected] Reply to: [email protected] Position: Lead Data engineer Location: Chicago, IL- oniste Need solid hands-on candidates. Coding test will be part of evaluation Need 14+ years of Exp. MUST HAVE: Python/Pyspark scripting Glue / lambda Snowflake (sql, stored procs) Airflow or equivalent orchestration AWS Cloud fomation Informatica/ETL Description: Design and implement data pipelines to extract, transform, and load data from various sources into a data warehouse Develop and maintain data models and schemas to support business intelligence and analytics applications Optimize data storage and retrieval systems for performance and scalability Collaborate with data scientists and analysts to develop and deploy machine learning models and algorithms Implement data governance policies and procedures to ensure data quality, security, and compliance Manage and monitor data processing and storage infrastructure, including cloud-based platforms such as AWS Develop and maintain ETL workflows using tools Design and implement data integration solutions for thirdparty applications and services Conduct performance tuning and optimization of database systems and queries Mentor and train junior data engineers and other team members on best practices and emerging technologies in data engineering. Building and managing DevOps & DataOps CI/CD pipelines and automated deployments Debugging and resolving issues with cloud service configuration, integration, performance, security standards, scalability. Prepare technical design documents and SLA agreements Ensure the delivery of thorough project documentation. Introduce best practices and provide necessary support from the Center of Excellence as needed. Manage the identification, evaluation, mitigation, and reporting of project risks. Set quality standards and perform regular quality reviews. Ensure that project issues and challenges are resolved promptly in collaboration with the client. Keywords: continuous integration continuous deployment Illinois Lead Data engineer - Chicago, IL- oniste [email protected] |
| [email protected] View all |
| Mon Oct 21 21:12:00 UTC 2024 |