| DATA ENGINEER || 12+ MONTH || IN PERSON || HYBRID at Remote, Remote, USA |
| Email: [email protected] |
| From: Jyoti Mittal, Kpg99 Inc [email protected] Reply to: [email protected] Hi, My name is Jyoti and I represent KPG99 INC. We work with clients across the USA. I have a very urgent role. If interested then please share with me your updated resume along with below information. we have managers requesting local Minneapolis talent (no relocation), Please see skill sets below and send me any qualified candidates. Data Engineer (Azure, Spark, Kafka experience required) USC with LI 2nd round in-person in Minneapolis, MN required 12+ month contracts Senior Software Engineer (Big Data - Spark) Primary Responsibilities: Design and build highly available, scalable, resilient and fault tolerant distributed real-time data streaming system. Automate test coverage (90+%) for data pipelines, best practices and frameworks for unit, functional and integration tests. Automate CI and deployment processes and best practices for the production data pipelines. Build AI/ML model based alert mechanism and anomaly detection system for the product. The goal is have a self-annealing product. Basic Qualifications Bachelors degree or equivalent work experience. 5+ years of relevant experience. Preferred Skills/Experience 10+ years of overall experience in software development with 5-6 years of relevant experience in designing, developing, deploying and operating data streaming pipelines at scale. 3-4 years experience with Apache Kafka and Apache Spark Streaming Hands on experience with Spark Structured Streaming Experience in tuning Spark Data pipeline to achieve high throughput. Drive efforts to improve the data quality across data pipelines and implement system controls for managing data quality Programming proficiency in Scala or Java. Open Source Committer (Apache Spark or related Big Data open source technologies) Experience with Containers, Kubernetes and scaling elastically Strong background in algorithms and data structures and continuously develop and acquire new technical skills Lead and mentor junior engineers to ensure systems are built with highest quality and leveraging best practices. Experience in automating Spark pipeline deployment/testing (DevOps, CI/CD) Passion for data engineering and for enabling others by making the product easier to use. Excellent communication in sharing context to effectively collaborate with analytical partners, domain experts and other consumers of your work, preferably in supporting an engineering or product function Required Skills Summary: Apache Spark, Apache Kafka, Scala/Java, NoSQL Databases, Elasticsearch & Kibana, Kubernetes, Docker Containers Thanks & Regards, Jyoti Mittal | Technical Recruiter | KPG99,INC [email protected] | http://www.kpg99.com/ 3240 E State St EXT, Hamilton, NJ 08619 An E- Verified Company Certified Minority Owned Business Note: KPG99 does not endorse undesired email. If you do not want to receive our mails, please reply with Remove in the subject; I would ensure that you are not troubled further. I apologize for any inconvenience. Keywords: continuous integration continuous deployment artificial intelligence machine learning Minnesota New Jersey DATA ENGINEER || 12+ MONTH || IN PERSON || HYBRID [email protected] |
| [email protected] View all |
| Mon Aug 05 21:56:00 UTC 2024 |