| Big Data Developer Need Local at Columbus, Ohio, USA |
| Email: [email protected] |
| From: Nitin Tehriya, Tek Inspirations LLC [email protected] Reply to: [email protected] Big Data Developer Location: Hybrid, Columbus, OH Need Local Duration: 6+month contract MOI: Skype The Technical Specialist will be responsible for Medicaid Enterprise Data Warehouse (EDW) design, development, implementation, migration, maintenance and operation activities. Works closely with Data Governance and Analytics team. The candidate will closely with Data Governance and Analytics team. Will be one of the key technical resource for data warehouse projects for various Enterprise Data Warehouse projects and building critical Data Marts, data ingestion to Big Data platform for data analytics and exchange with State and Medicaid partners. This position is a member of Medicaid ITS and works closely with the Business Intelligence & Data Analytics team. Required skill sets: 9+ years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects. Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Cloudera Bigdata Technologies (with a minimum of 8-9 years experience in Hadoop, MapReduce, Sqoop, PySpark, Spark, HDFS, Hive, Impala, StreamSets, Kudu, Oozie, Hue, Kafka, Yarn, Python, Flume, Zookeeper, Sentry, Cloudera Navigator) along with Oracle SQL/PL-SQL, Unix commands and shell scripting; Strong development experience (minimum of 8-9 years) in creating Sqoop scripts, PySpark programs, HDFS commands, HDFS file formats (Parquet, Avro, ORC etc.), StreamSets pipeline creation, jobs scheduling, hive/impala queries, Unix commands, scripting and shell scripting etc. Writing Hadoop/Hive/Impala scripts (minimum of 8-9 years experience) for gathering stats on table post data loads. Strong SQL experience (Oracle and Hadoop (Hive/Impala etc.)). Writing complex SQL queries and performed tuning based on the Hadoop/Hive/Impala explain plan results. Experience building data sets and familiarity with PHI and PII data. Expertise implementing complex ETL/ELT logic. Accountable for ETL/ELT design documentation. Good knowledge of Big Data, Hadoop, Hive, Impala database, data security and dimensional model design. Basic knowledge of UNIX/LINUX shell scripting. Utilize ETL/ELT standards and practices towards establishing and following centralized metadata repository. Participate in ETL/ELT code review and design re-usable frameworks. Create Remedy/Service Now tickets to fix production issues, create Support Requests to deploy Database, Hadoop, Hive, Impala, UNIX, ETL/ELT and SAS code to UAT environment. Create Remedy/Service Now tickets and/or incidents to trigger Control M jobs for FTP and ETL/ELT jobs on ADHOC, daily, weekly, monthly and quarterly basis as needed. Model and create STAGE / ODS / Data warehouse Hive and Impala tables as and when needed. Familiar with Project Management methodologies like Waterfall and Agile Ability to establish priorities & follow through on projects, paying close attention to detail with minimal supervision. Required Education: BS/BA degree or combination of education and experience. Create re-usable framework for Audit Balance Control to capture Reconciliation, mapping parameters and variables, serves as single point of reference for workflows. Create PySpark programs to ingest historical and incremental data. Create SQOOP scripts to ingest historical data from EDW Module Vendors databases to Hadoop IOP, created HIVE tables and Impala views creation scripts for Dimension tables. Create jobs in Hadoop using SQOOP, PYSPARK and Stream Sets to meet the business user needs. Keywords: business analyst procedural language Ohio |
| [email protected] View all |
| Tue Dec 12 21:23:00 UTC 2023 |