| Databricks Architect at Remote, Remote, USA |
| Email: [email protected] |
| From: Manoj, Nitya Software Solutions [email protected] Reply to: [email protected] Databricks Architect Eden Prairie, MN Key Responsibilities: Work on client projects to deliver Microsoft Azure based Data engineering & Analytics solutions Provide hands on technical leadership for relevant technical skills on multiple projects if required Engineer and implement scalable analytics solutions on Azure (focusing on Databricks) Design pipelines to ingest & integrate structured/unstructured data from heterogeneous sources Collaborate with delivery leadership to deliver projects on time adhering to the quality standards Contribute to the growth of the Microsoft Azure practice by helping with solutioning for prospects Contribute to talent recruitment, competency building and providing mentor-ship to junior resources Mandatory Skills Over all 13 + Years experience and a Minimum 3+Years exp in Azure, should have worked as an architect for at least 3 years, 3+ Data Modelling experience Should be able to Define and communicate data architecture requirements, keeping current with data management best practices. Should come from DWH background, should have strong ETL experience Strong hands on exp. of ADF, Azure Data Bricks/Pyspark Worked on new databricks features like unity catalog, DLT, etc Well versed with CI/CD Strong Knowledge of Azure EventHubs and Pub-Sub model, security Highly proficient at SQL development Experience in BAU projects in Azure space should be able to bring in best practices and stabilize the environment Collaborates with data owners to establish data quality rules and definitions. Implements the data quality rules in a data management system. Work as team lead to develop Cloud Data and Analytics solutions Participate in the development of cloud data warehouses, data as a service, business intelligence solutions Data wrangling of heterogeneous data Coding complex Spark (Scala or Python). Developing Modern Data Warehouse solutions using Azure Stack (Azure Databricks) In-depth knowledge Data processing techniques and handling large data sets Hands on experience using Azure Blob storage and Azure Data Lake Services for storing data Hands on experience using Databricks in at least 2 projects, creating notebooks using Scala/Python Exposure to using Pyspark, Spark SQL & Spark streaming for processing data for various use cases Good understanding of Delta lake and how it can be used in conjunction with Blob or ADLS Experience using Azure Synapse as source or target for the data pipelines in at least 1 project Knowledge of Azure Data Factory or Databricks scheduling features to automate Databricks jobs Problem-solving skills along with good interpersonal & communication skills Self-starter who can pick up any other relevant Azure Services in the Analytics space Keywords: continuous integration continuous deployment information technology Minnesota |
| [email protected] View all |
| Thu Oct 05 01:17:00 UTC 2023 |