| Sr. Data Engineer at Remote, Remote, USA |
| Email: [email protected] |
| From: vivek kumar, vyzeinc [email protected] Reply to: [email protected] Title: Sr. Data Engineer Location: Remote Contract: 6-month C2H Moi: Skype Skills: python, spark, AWS, Kafka, AWS Glue, client does NOT want to see candidates with Scala LinkedIn Must !! Healthcare domain candidate Duties and Responsibilities: The Senior Data Engineer is responsible for creating data acquisition strategy and develops data set processes. He or she will be also responsible for finding trends in datasets and developing workflows and algorithms to help make raw data more useful to the enterprise. Leading design, implementation, and ongoing management of robust, scalable, and flexible data pipelines supporting critical data and application integration Building and scaling the distributed infrastructure that drives the Amazon's EMR platform Automating workflows leveraging DevOps framework (CI/CD/CT) where applicable Ensuring quality of technical solutions as data moves across Healthfirst environments Working with solution and data architects to develop, construct, test and maintain architectures, prototypes, and design solutions Coach and mentor junior technical staff for instance, but not limited to, developers and testers Helping to maintain the integrity and security of the company data Providing insight into the changing data environment, data processing, data storage and utilization requirements for the company and offer suggestions for solutions Articulating both the technical implications as well as data usage implications of proposed solutions or solution options to a wide variety of stakeholders. Working directly with business users to align solutions with business requirements Relentless focus on continuous improvement of solutions in to improve data reliability, efficiency, quality and more Working with solution and enterprise architects to conduct research for industry best practices and developments and recommend new features and/or improvements to existing practices and design. Create data monitoring capabilities for each business process and work with data consumers on updates Coaching peers and other members of the team through new solutions, concepts, and technologies Diagnosing and resolving questions and issues about the data to ensure data usability for the consumers Minimum Qualifications: Firm foundation in software development principles and practices 5+ years experience in a building robust data pipelines/ETL and data processing 3+ years' experience in building and scaling the distributed infrastructure of Amazon's EMR platform 3+ years' hands-on experience with Pyspark/Spark SQL 3+ years experience working in a production cloud infrastructure. Prior backgrounds with AWS services especially S3, Lambda, and EMR services Proficiency in SQL a must - knowledge of SQL and multiple programming languages in order to optimize data processes and retrieval. 5+ years experience is required. Experience working in a Big Data ecosystem processing and building reliable and scalable solutions involving file systems, data structures/databases, automation, security, messaging, integration etc. Strong knowledge in the concepts of RDBMS/Cloud database services such as AWS RDS PostgreSQL, Redshift, Snowflake or Dynamo DB Exposure to CI/CD/CT for software systems Such as Jenkins, Artifactory and CloudFormation Must be able to develop creative solutions to problems Demonstrates critical thinking skills with ability to communicate across functional departments to achieve desired outcomes Excellent interpersonal skills with proven ability to influence with impact across functions and disciplines. Ability to work independently and as part of a team Ability to manage multiple projects/deadlines, identifying the necessary steps and moving forward through completion HS Diploma or GED from an accredited institution Preferred Qualifications: Bachelors Degree from an accredited institution Leadership Capabilities with a proven track record of success directing the efforts of data engineers and business analysts within a deadline-driven and fast-paced environment Demonstrated experience working in an Agile environment as a Software Engineer focused on data processing 3+ years of working with messaging/data streaming services especially Kafka/ Amazon MKS 3+ years experience in programing languages such as C/C++, Java, Python, etc. Strong expertise with Linux environment preferred Knowledge of provider-sponsored health insurance systems/processes and the Healthcare industry Keywords: cprogramm cplusplus continuous integration continuous deployment sthree database Connecticut |
| [email protected] View all |
| Mon Jul 17 21:06:00 UTC 2023 |