Home

Hiring Now :: Unity Catalog Engineer :: Remote at Unity, Wisconsin, USA
Email: [email protected]
From:

Amit Kumar Pradhan,

Vyze Inc.

[email protected]

Reply to: [email protected]

Job Description -

Job Title: Unity Catalog Engineer

Location: 100 % Remote

Duration: 12+ months

Visa: USC/GC/H4EAD (Via copy required Front & Back copy)

Moi: Skypec2c

Responsibilities:
Collaborate closely with clients to design and customize the Unity Catalog structure according to their specific requirements and workflows.
Develop mappings of Metastore artifacts to Unity Catalog Catalogs and Schemas, ensuring seamless integration and data consistency.
Lead the upgrade process for tables, clusters, and workflows, migrating 293 tables and 100 jobs to the upgraded Databricks Unity Catalog.
Prepare continuous integration and continuous deployment (CI/CD) pipelines for managing code changes in jobs.
Ensure data quality and consistency throughout the migration process, optimizing data structures and configurations for enhanced performance and scalability.
Establish robust rollback procedures to address any migration issues effectively.
Synchronize data to mapped schemas to maintain data integrity and coherence.

QA, Enablement, and Cutover:
Conduct comprehensive testing of upgraded Databricks Unity Catalog objects to validate functionality and reliability.
Conduct training sessions for end-users to familiarize them with Unity Catalog usage and best practices for metadata management.
Provide hands-on training for the internal IT team on catalog management processes and procedures.
Share expertise and insights on metadata management best practices to optimize system performance and user experience.

Skills and Qualifications:
Bachelor's degree in Computer Science, Engineering, or a related field.
7+ years of experience in data engineering or related roles, with a focus on metadata management and data catalog solutions.
Proficiency in designing and implementing complex data structures and workflows using Databricks Unity Catalog or similar platforms.
Strong expertise in data migration processes, including mapping, transformation, and validation.
Experience with continuous integration and continuous deployment (CI/CD) practices and tools.
Excellent problem-solving skills and attention to detail, with a focus on ensuring data quality and integrity.
Strong communication and interpersonal skills, with the ability to collaborate effectively with clients and internal teams.
Proven track record of delivering high-quality solutions on time and within budget.

Keywords: continuous integration continuous deployment quality analyst information technology green card
Hiring Now :: Unity Catalog Engineer :: Remote
[email protected]
[email protected]
View all
Tue May 21 09:33:00 UTC 2024

To remove this job post send "job_kill 1412630" as subject from [email protected] to [email protected]. Do not write anything extra in the subject line as this is a automatic system which will not work otherwise.


Your reply to [email protected] -
To       

Subject   
Message -

Your email id:

Captcha Image:
Captcha Code:


Pages not loading, taking too much time to load, server timeout or unavailable, or any other issues please contact admin at [email protected]
Time Taken: 26

Location: , Remote