| Urgent Profile needed - Databricks Lead. - Chicago, IL - Hybrid at Chicago, Illinois, USA |
| Email: [email protected] |
| From: John Joseph, IDC Technologies.com [email protected] Reply to: [email protected] Must Have Bachelors degree in Computer Science, a related field, or applicable work experience. 7+ years of overall IT experience with 5+ years of experience in Databricks platform development & administration, Spark and cloud experience in AWS, Azure or GCP cloud (GCP preferred).Primary skills: Databricks administration & development Databricks Resource Management Skills and Expertise In-depth understanding of Big Data ecosystems underlying architectures to re-platform legacy systems and build integration solutions on Google Cloud platform. Experience in design and development of machine learning pipelines to process structured and unstructured data streams. Hands on experience developing complex services/solutions on Databricks Platform. Expertise in Java, Python or any scripting language, SQL/NoSQL Database systems. Experience in Big Data tools and technologies such as Big Query, Databricks, Snowflake, Spark, Kafka Streams Have a passion for new technologies, continuous improvement and actively involved with the team to design new data solutions, monitoring and alerting policies for data solutions. Experience in test automations ensuring data quality, continuous integrations build and deployment processes using Docker, GitHub, Jenkins and Unix/Linux shell scripts. Design and implement frameworks and data pipelines for large-scale data processing on Databricks that will power data science capabilities across the enterprise. Develop MLOps frameworks which will enable model development, training, deployment, and monitoring. Configure and manage Databricks clusters ensuring optimal performance, scalability, and resource utilization. Manages user provisioning and access controls within the Databricks environment. Monitor cluster health, performance metrics, and resource consumption. Scale clusters based on workload demands. Troubleshoot and resolve issues related to connectivity, and performance bottlenecks. Manage Databricks workspaces, including creating notebooks, folders, and organizing resources for different teams and projects. Keywords: information technology |
| [email protected] View all |
| Fri Oct 06 00:08:00 UTC 2023 |