| Tomorrow Interview Slots -- Data Architect with Data Bricks -- Onsite -- Raritan, NJ -- PP no. is Mandatory. at Raritan, New Jersey, USA |
| Email: [email protected] |
| Hi, I hope this email finds you well. I am working on the Following Job Opening. Please have a look and let me know if you are Interested in this role. Please share only local or nearby profile's to NJ/NY/PA. OPT / CPT / H1T Candidate's won't work. Role: Databricks Architect. Location: Raritan, NJ (On-site). Long Term Contract. Job Description: -- Solution Design: Databricks architects are responsible for designing data pipelines and workflows that leverage Databricks to process and analyze large volumes of data. They design solutions that meet the business requirements and ensure scalability, performance, and reliability. -- Data Ingestion: Architects design strategies for ingesting data from various sources, such as databases, data lakes, and streaming platforms, into Databricks. This involves selecting the appropriate data connectors and integration technologies. -- Data Transformation: They create data transformation and processing pipelines using Apache Spark, SQL, and other Databricks libraries. These pipelines are designed to clean, preprocess, and transform raw data into a suitable format for analysis. -- Data Storage: Databricks architects determine the optimal data storage solutions for their use cases, which may include using Databricks Delta Lake, data lakes, data warehouses, or other storage technologies. -- Security and Compliance: Ensuring data security and compliance with regulations is a critical aspect of the role. Databricks architects design security measures, access controls, and encryption for data at rest and in transit. -- Performance Optimization: They work on optimizing the performance of Databricks workloads, including tuning Spark configurations, optimizing queries, and designing efficient data partitioning strategies. -- Integration with Machine Learning: Databricks is also widely used for machine learning and AI projects. Architects design pipelines for training and deploying machine learning models within Databricks. -- Monitoring and Maintenance: Monitoring the health and performance of Databricks clusters and pipelines is essential. Architects set up monitoring and alerting systems to ensure that issues are promptly addressed. -- Collaboration and Best Practices: Databricks architects collaborate with data engineers, data scientists, and analysts to ensure that best practices are followed in data architecture and analysis. They also provide guidance and support to the data team. -- Cost Management: Architects need to be conscious of cost management and resource optimization, as Databricks is a cloud-based platform, and efficient resource allocation can lead to cost savings. -- Keywords: artificial intelligence access management information technology New Jersey New York Pennsylvania |
| [email protected] View all |
| Tue Oct 24 19:04:00 UTC 2023 |