| Onsite - Data Architect - Databricks - Need 15+ year profile at Remote, Remote, USA |
| Email: [email protected] |
| Experience with Insurance Domain , Must have experience with Guidewire Position : Data Architect - Databricks Location : Bloomington IL Long Term , Any visa is ok, non local is ok Experience with Insurance Domain , Must have experience with Guidewire. 15+ yearsof experience in the IT/Technology and having at least 3+ years of experience in Azure/Databricks Design and architect Databricks-based solutions that align with business objectives, ensuring scalability, performance, and security. Designed, Development and implemented Datamesh using DeltaLake. Provide technical leadership and guidance to the Databricks development team, ensuring best practices are followed throughout the project lifecycle. Collaborate with cloud infrastructure teams to design and optimize the underlying infrastructure for Databricks workloads on platforms such as AWS or Azure Develop efficient data ingestion and ETL pipelines using Databricks, Apache Spark, and other relevant technologies. Integrate Databricks with data lakes and data warehouses to ensure seamless data access and analytics. Continuously monitor and optimize Databricks workloads for performance and cost-effectiveness. Implement and maintain security measures, including access controls, encryption, and compliance standards, to protect sensitive data. Create documentation and provide training to internal teams on Databricks best practices and usage. Stay up-to-date with the latest developments in Databricks and related technologies to recommend and implement improvements. Good knowledge on SQL Queries & Stored Procedures. Strong programming skills in Pyspark, Python, writing complex queries in T-SQL Experience in transferring data from RDBMS to Data bricks using ADF. Expertise in using Spark SQL with various data sources like JSON, Parquet and XML. Experience in creating tables, partitioning, bucketing, loading and aggregating data using Spark SQL/Python -- Keywords: information technology Illinois Onsite - Data Architect - Databricks - Need 15+ year profile [email protected] |
| [email protected] View all |
| Wed Apr 17 21:01:00 UTC 2024 |