| Data Bricks Data Engineer Missouri onsite at Remote, Remote, USA |
| Email: [email protected] |
| From: shaik, Convextech [email protected] Reply to: [email protected] Title Data Bricks - Data Engineer Duration 9+ Months Location O'Fallon, Missouri (Day 1 onsite) Job Description : Minimum 6+ years of IT experience and minimum 2+ years of hands-on experience in Databricks with validated knowledge in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts along with following: o Familiarity with Lakehouse Architecture with Delta Tables Delta tables, Schema Evolution, ACID compatibility, History capture etc. o Integrating Databricks with various ETL/Orchestration tools and perform required operations. o Python is Must o CI/CD Pipleine with AWS o SQL and SPARK with Date Engineering background o Expertise in Spark ecosystems, Data frame API, Data set API, RDD APIs & Spark SQL. o Expertise in workspace and cluster configuration, performance tuning and log monitoring. o Experience with structured streaming along with streaming frameworks. o Hands on experience with Python, Scala and SQL. o Create and maintain optimal data pipeline architecture with knowledge in best practices from Databricks around Cluster setup/Development/Deployment o Strong understanding of Data Modelling and defining conceptual logical and physical data models. o Experience in building Data Engineering solutions using AWS/Azure native stack o Experience in managing various client relationships to get consensus on solutions. o Nice to have AI/ML skill and any architect level certification from Databricks Keywords: continuous integration continuous deployment artificial intelligence machine learning information technology |
| [email protected] View all |
| Wed Sep 13 19:51:00 UTC 2023 |