| Urgent Requirement || Data Engineer with Python + AWS + Spark || Plano, TX(lical only) || c2c at Plano, Texas, USA |
| Email: [email protected] |
| Job Title: Data Engineer with Python + AWS + Spark Location : Plano, TX (Local Only) Experience 9+ Years AWS - 40% Python - 30% Spark - 30% Position summary A Data Engineer at CMS is a software engineer with proficiency in data. The data engineer will build and maintain the CMS data warehouse which is used for both reporting and analytics across the company. The individual works cross functionally with technical and business teams to identify opportunities to better leverage data. The data comes from a variety of sources and it is the responsibility of the data engineer to make sense of the data using cloud based systems (AWS) and provide a reliable and structured format to meet the different business needs at CMS. Duties and responsibilities Collaborate with the team to build out features for the data platform and consolidate data assets Build, maintain and optimize data pipelines built using Spark Advise, consult, and coach other data professionals on standards and practices Work with the team to define company data assets Migrate CMS data platform into Chases environment Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives Build libraries to standardize how we process data Loves to teach and learn, and knows that continuous learning is the cornerstone of every successful engineer Has a solid understanding of AWS tools such as EMR or Glue, their pros and cons and is able to intelligently convey such knowledge Implement automation on applicable processes The ideal candidate 5+ years of experience in a data engineering position Proficiency is Python (or similar) and SQL Strong experience building data pipelines with Spark Strong verbal & written communication Strong analytical and problem solving skills Experience with relational datastores, NoSQL datastores and cloud object stores Experience building data processing infrastructure in AWS Bonus: Experience with infrastructure as code solutions, preferably Terraform Bonus: Cloud certification Bonus: Production experience with ACID compliant formats such as Hudi, Iceberg or Delta Lake Bonus: Familiar with data observability solutions, data governance frameworks Requirements Bachelors Degree in Computer Science/Programming or similar is preferred Right to work Best Regards www.diverselynx.com Sapna Verma E-Mail: [email protected] Diverse Lynx LLC, 300 Alexander Park, Suite 200. Princeton, NJ 08520. -- Keywords: information technology New Jersey Texas Urgent Requirement || Data Engineer with Python + AWS + Spark || Plano, TX(lical only) || c2c [email protected] |
| [email protected] View all |
| Thu Oct 17 22:33:00 UTC 2024 |