| Trying to reach you for Databricks Administrator - Foster City, CA at Foster City, California, USA |
| Email: [email protected] |
| From: Ajay, Yochana IT [email protected] Reply to: [email protected] Hi , I hope you are doing well and having a great day! I wanted to reach out to you regarding a Contract job opportunity as a Databricks Administrator Foster City, CA. My client has a strong reputation for being one of the leading employers in the industry. After reviewing your qualification and resume, I feel you would be a good fit for this role. I have listed some of the main requirements for this position below for your review. Position Name Databricks Administrator Type of hiring Contract Location Foster City, CA (Onsite) Required AWS, Kafka, Databricks, Admin Job Description: We are seeking a highly skilled and experienced individual to join our IT organization in the role of Databricks Administrator. This individual will play a crucial role in leading a centralizing project focused on Databricks administration within our organization. The successful candidate will have extensive expertise in AWS, Kafka, and Databricks administration, along with a strong understanding of data management principles. Responsibilities: Lead Databricks Centralization Project: Take charge of the project aimed at centralizing Databricks administration within our IT organization. Develop a comprehensive strategy and roadmap for migrating, consolidating, and managing Databricks workloads efficiently. AWS Infrastructure Management: Work with IT infrastructure leaders to manage components on AWS to support Databricks workloads effectively. Kafka Administration: Understanding of how to handle topics, partitions, producers, consumers, and other Kafka components as needed for Databricks integration. Databricks Administration: Configure, monitor, and maintain Databricks clusters, workspaces, jobs, notebooks, libraries, and integrations with other AWS services. Implement best practices for security, performance tuning, and cost optimization. Automation and Scripting: Develop automation scripts and workflows using tools like Python, Shell scripting, Terraform, or AWS CloudFormation to streamline provisioning, deployment, and management tasks for Databricks and associated resources. Monitoring and Performance Tuning: Implement monitoring solutions and performance tuning strategies to ensure the optimal operation of Databricks workloads. Proactively identify and address performance bottlenecks, resource contention issues, and other operational challenges. Security and Compliance: Implement and enforce security policies, access controls, encryption mechanisms, and data governance practices to protect sensitive data and ensure compliance with regulatory requirements (e.g., GDPR, HIPAA). Collaboration and Knowledge Sharing: Collaborate with cross-functional teams including data engineers, data scientists, and business stakeholders to understand requirements, provide technical guidance, and support Databricks-related initiatives. Share knowledge and best practices through documentation, training sessions, and workshops. Qualifications: Bachelor's Degree in Computer Science, Information Technology, or related field. Ownership of Databricks as a strategic and operational resource - Complete ownership of the product for the data solutions Expertise in migrating to and enabling Unity Catalog (UC) within Databricks environments. Strong proficiency in Kafka administration, with hands-on experience managing Kafka clusters, topics, partitions, and consumer/producer configurations. In-depth knowledge of Databricks administration, including cluster management, notebook development, job scheduling, and library management. Management of Databricks at the Workspace and Account levels with a centralized approach. Experience with setting up UC across multiple workspaces, including separate environments for Production and Non-Production. Capability to handle deduplication of workspaces efficiently. Proficient in implementing access control mechanisms to secure data and workflows. Ability to assist during the migration to Unity Catalog, ensuring a smooth transition. Knowledge and experience in managing licensing, resource utilization, scaling, and storage specifically for Databricks on AWS. Experience in spinning up new Databricks workspaces and migrating data from AWS S3 buckets. Ability to create and manage new server clusters on AWS efficiently. Good Experience in Apache Spark and Delta Lake Technology (DLT), as well as Hive Metastore management. Skills in query optimization, managing notebooks, and effectively utilizing Databricks for data operations. Understanding of role-based access control (RBAC) and attribute-based access control (ABAC), along with overall data governance. Expertise in metadata modeling to support effective data management and analysis. Knowledge in handling sensitive information, including skills in data segregation, encryption, and anonymization to ensure privacy and compliance. Basic knowledge of AWS services, including VPC, Private Subnet, NACL, and VPC Endpoints Proficiency in scripting and automation using Python, Shell scripting, or similar languages. Experience with infrastructure-as-code tools like Terraform or AWS CloudFormation is a plus. Solid understanding of security principles and best practices in cloud environments, particularly AWS IAM, encryption, and network security. Experience in performance tuning, monitoring, and troubleshooting of distributed systems and big data platforms. Excellent communication skills with the ability to collaborate effectively with technical and non-technical stakeholders. Certifications such as AWS Certified Solutions Architect, AWS Certified DevOps Engineer, or Databricks Certified Associate are desirable but not required. Thanks & Regards, Ajay Yochana IT Solutions INC , 23000 Commerce Dr, Farmington hills, MI-48335 Keywords: sthree information technology California Michigan Trying to reach you for Databricks Administrator - Foster City, CA [email protected] |
| [email protected] View all |
| Tue May 07 04:23:00 UTC 2024 |