| Databricks Engineer with PySpark, Python - OFallon ,MO - Onsite at ofallon, Missouri, USA |
| Email: [email protected] |
| From: anuradha, Talent Groups [email protected] Reply to: [email protected] Hello, Hope you are doing well. My name is Anuradha, and I represent Talent Group, a staff augmentation firm providing a wide-range of talent on-demand and total workforce solutions. We have excellent domain expertise in all verticals. Repositioning professionals is what we do, and we do it very well. Please look at the job description below and let me know your interest. Role: Databricks Engineer with PySpark, Python Location: OFallon ,Mo (Onsite) Contract Job Description: Responsibilities: Design, Develop and deploy ETL processes and data pipelines using Databricks Implement data ingestion from various sources into data bricks, ensuring data quality and reliability Build, Optimize and fine-tune Databricks workflows to improve performance and reliability Must have skills: 10+ years of programming experience in PySpark, Python and SQL Experienced in designing and implementing complex and scalable Data pipelines / ETL using Databricks. Experienced with cloud-based data storage and processing technologies, particularly AWS services(S3, Step functions, Lambda, Airflow) Familiarity with CI/CD practices, version control (Git), automated testing and Agile environment Hands-on experience writing complex SQL queries, exporting, and importing large amounts of data using utilities . Nice to have skills: knowledge on Big data Hadoop Experience with structured streaming along with streaming frameworks. Ensure compliance with data security and privacy standards Certification in Databricks Experience in Hadoop Migration to Databricks. Keywords: continuous integration continuous deployment sthree information technology Missouri Databricks Engineer with PySpark, Python - OFallon ,MO - Onsite [email protected] |
| [email protected] View all |
| Tue Jul 02 04:23:00 UTC 2024 |