| Lead Data Engineer at Remote, Remote, USA |
| Email: [email protected] |
| From: Abhishek, heliogic [email protected] Reply to: [email protected] Title : Lead Data Engineer (No GC visa) Long term Contract Location : Remote Exp 10 - 15 years Mandatory passport number and linkedin should be joined before 2019 Responsibilities: Collaborate with machine learning practitioners to understand feature and data requirements Work with Engineering teams to collect required data from internal and external systems Develop and maintain production-grade batch and streaming pipelines Setup orchestration and automation of ETL jobs using tools such as Airflow and Jenkins Improve and extend existing data infrastructure services Conduct data exploration, analysis and provide data strategy consultancy Drive and maintain a culture of quality, innovation and experimentation Work in an Agile environment that focuses on collaboration and teamwork Mentor colleagues on best practices and technical concepts of building large scale solutions Qualifications: 8+ years of data engineering experience Experience deploying and running services in AWS and in engineering big-data solutions using technologies like Databricks, EMR, S3, and Spark Experience building streaming pipelines using Kafka, Spark, Flink, or Samza Experience loading and querying cloud-hosted databases such as Redshift and Snowflake Experience designing and developing backend microservices for large scale distributed systems using gRPC or REST Experience with graph-based data workflows such as Apache Airflow, Meson Excellent communication and people engagement skills In short, we need streaming data engineer Lead : Build tooling and low-latency services to enable and support event-driven data pipelines Expert in spark scala, python, sql Experience in building large datasets and scalable services Experience building streaming data solutions using technologies like Databricks, Flink, Spark Keywords: sthree green card Lead Data Engineer [email protected] |
| [email protected] View all |
| Wed Apr 17 18:22:00 UTC 2024 |