| New Requirement Need :: 8+ Data Engineer with Data Bricks at ofallon, Missouri, USA |
| Email: [email protected] |
| Title - Data Bricks - Data Engineer Location O'Fallon , Missouri (Day 1 onsite) Contract Job Description Minimum 5+ years of IT experience and minimum 3+ years of hands-on experience in Databricks and pyspark with validated knowledge in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts along with following: Familiarity with Lakehouse Architecture with Delta Tables Delta tables, Schema Evolution, ACID compatibility, History capture etc. Integrating Databricks with various ETL/Orchestration tools and perform required operations. Python is Must CI/CD Pipeline with AWS SQL and SPARK with Date Engineering background Expertise in Spark ecosystems, Data frame API, Data set API, RDD APIs & Spark SQL. Expertise in workspace and cluster configuration, performance tuning and log monitoring. Experience with structured streaming along with streaming frameworks. Hands on experience with Python, Scala and SQL. Create and maintain optimal data pipeline architecture with knowledge in best practices from Databricks around Cluster setup/Development/Deployment Strong understanding of Data Modelling and defining conceptual logical and physical data models. Experience in building Data Engineering solutions using AWS native stack Experience in managing various client relationships to get consensus on solutions. Nice to have AI/ML skill and any architect level certification from Databricks -- Keywords: continuous integration continuous deployment artificial intelligence machine learning information technology New Requirement Need :: 8+ Data Engineer with Data Bricks [email protected] |
| [email protected] View all |
| Wed Apr 17 21:10:00 UTC 2024 |