| Big Data Architect at Remote, Remote, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=232691&uid= From: John, Fluxtek [email protected] Reply to: [email protected] Position: Big Data Architect Location: Remote Duration: 6+ Months Exp. Req: 12+ Job Description- Key Roles and Responsibilities o Build, Test, Deploy, and troubleshoot Data Pipelines o Build and review data pipelines for data ingestion, data cleansing, aggregation, and readying data for consumption o Work on integration and deployment tools, and methods to manage the dev life cycle until production o Build the infrastructure, and data pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, and AWS big data technologies o Work on integration and deployment tools, and methods to manage the dev life cycle until production o Collaborate with Lead engineers, and owns to deliver solutions o Participate in conversations and provide technical solutions on escalated production, performance, and availability issues Technical Skills o Experience Exposure and working knowledge on building PySpark applications reading data from Postgres & MongoDB Implementation of PySpark applications in AWS EMR, and ingesting data into S3 Exposure and experience on Java based Big Data pipelines Involved in migrating ETL processes, preferably Cloudera ETL processes to PySpark Nice to have knowledge of Map Reduce, Hive Exposure and working knowledge of using Databricks to run PySpark applications Exposure and implemented job orchestration using Apache Airflow Knowledge of encryption and decryption of sensitive data Knowledge of Terraform to enable infrastructure as code o Nice to have experience or working knowledge with data pipeline and workflow management tools: Octopus, Jenkins, Gitlab, Git o Experience with AWS cloud services: EMR, RDS, S3 o Experience building and optimizing big data pipelines, architectures, and data sets o Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement o Build processes supporting data transformation, data structures, metadata, dependency, and workload management o A successful history of manipulating, processing, and extracting value from large datasets o Good to have a working knowledge of message queuing, stream processing, and highly scalable big data stores Additional details: o Programming Languages Python, Java /Scala o Big Data technologies HDFS, Cloudera ETL, PySpark Databricks Apache Airflow o Databases Postgres, MongoDB o Cloud technologies AWS, S3, EMR Thanks & Regards..... John Chinthala Technical Recruiter - Fluxtek Solutions Inc D : 832 - 361 - 3756 Ext : 258 Whatsapp :+1 (832) 558 - 4407 E: [email protected] https://www.linkedin.com/in/john-sung-chinthala-7b6b9523a/ Certified Woman-Owned Business Enterprise (WMBE) Keywords: sthree http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=232691&uid= |
| [email protected] View All |
| 06:02 PM 23-Dec-22 |