| Big Data Engineer, Ridgefield Park, NJ (Hybrid Onsite) at Ridgefield, Washington, USA |
| Email: [email protected] |
| From: abhishek, StellentIT [email protected] Reply to: [email protected] Big Data Engineer, Ridgefield Park, NJ 12+ Months Phone + Skype Big Data Engineer, Ridgefield Park, NJ 12+ Months Phone + Skype Job Responsibilities Big Data Engineer: Top Skill Sets: SQL(Hive), Python, Spark Responsibilities include: Skills Required: 5-8 years of experience writing complex SQL queries. 4+ years of Python development experience 5+ years of demonstrated technical proficiency with Hadoop and big data projects 3+ years of demonstrated experience and success in data modeling. Ability to automate and implement ETLs to reduce manual work. Fluent in writing shell scripts [bash, korn] Writing high-performance, reliable and maintainable code. Knowledge and ability to implement workflow/schedulers within Oozie Experience working with AWS components Analytical and problem solving skills, applied to Big Data domain Proven understanding and hands on experience with Hadoop, Hive, Presto and Spark Good aptitude in multi-threading and concurrency concepts. B.S. or M.S. in Computer Science or Engineering Duties Translate complex functional and technical requirements into detailed design. Design for now and future success Hadoop technical development and implementation. Loading from disparate data sets. by leveraging various big data technology e.g. Kafka Pre-processing using Hive, Impala, Spark, and Pig Design and implement data modeling Maintain security and data privacy in an environment secured using Kerberos and LDAP High-speed querying using in-memory technologies such as Spark. Following and contributing best engineering practice for source control, release management, deployment etc Production support, job scheduling/monitoring, ETL data quality, data freshness reporting Keywords: New Jersey |
| [email protected] View all |
| Wed Jul 26 22:28:00 UTC 2023 |