| AWS BIG DATA Engineer |||||| Please share local to Columbus,OH /Plano,TX / Wilmington,DE only ||Please share H1B with passport number at Columbus, Ohio, USA |
| Email: [email protected] |
| AWS BIG DATA Engineer Columbus OH, Plano TX, Wilmington DE TCS Job description Role and responsibilities : - Creating Project Technical Documentation - Designing Solution architecture, and work on Data Ingestion, Preparation and Transformation. Debugging the production failures and identifying the solution. Developing efficient frameworks for development and testing using (AWS Dynamo DB, EKS, Kafka) to enable seamless data ingestion process on to the Hadoop platform. Enabling Data Governance and Data Discovery on Hadoop Platform Building data processing framework using Spark, HQL Exposure of Security Framework with Kerberos, Ranger, Atlas Exposure of Data Pipeline Automation using DevOps tools Exposure of Job Monitoring framework along validations automation - Exposure of handling structured, Un Structured and Streaming data technical skills requirements : - Solid hands-on and Solution Architecting experience in Big-Data Technologies (AWS preferred) Hands on experience in: AWS Dynamo DB, EKS Kafka, Kinesis, Glue, EMR Hands-on experience of programming language like Scala with Spark. Good command and working experience on Hadoop Map Reduce, HDFS, Hive, HBase, and/or No-SQL Databases Hands on working experience on any of the data engineering analytics platform (Hortonworks Cloudera MapR AWS), AWS preferred Hands-on experience on Data Ingestion Apache Nifi, Apache Airflow, Sqoop, and Oozie Hands on working experience of data processing at scale with event driven systems, message queues (Kafka FlinkSpark Streaming) Hands on working Experience with AWS Services like EMR, Kinesis, S3, CloudFormation, Glue, API Gateway, Lake Foundation - Hands on working Experience with AWS Athena - Data Warehouse exposure on Apache Nifi, Apache Airflow, Kylo - Operationalization of ML models on AWS (e.g. deployment, scheduling, model monitoring etc.) Feature Engineering Data Processing to be used for Model development Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.) Experience building data pipelines for structured unstructured, real-time batch, events synchronous asynchronous using MQ, Kafka, Steam processing Hands-on working experience in analyzing source system data and data flows, working with structured and unstructured data Must be very strong in writing SQL queries Strengthen the Data engineering team with Big Data solutions Strong technical, analytical, and problem-solving skills -- Keywords: machine learning message queue sthree database information technology Delaware Ohio Texas AWS BIG DATA Engineer |||||| Please share local to Columbus,OH /Plano,TX / Wilmington,DE only ||Please share H1B with passport number [email protected] |
| [email protected] View all |
| Fri May 17 19:08:00 UTC 2024 |