| Data Engineer (Streaming Platforms ) at Remote at Remote, Remote, USA |
| Email: [email protected] |
| From: JUNAID, Smartfolks [email protected] Reply to: [email protected] Hello! Hope you are well, we are looking for Data Engineer (Streaming Platforms ) this is remote role. Please refer JD below and share suitable profiles, looking forward to your response! Role: Data Engineer (Streaming Platforms ) Location: Remote customer is doing near real-time data integration. They do streaming analytics and they emphasize on Live Table Structured streaming, Kafka, Event Hub, Apache Spark. These technologies are definitely required to do near real-time data integration. Job Description : Streaming Platforms Mandatory BS, MS, or PhD in Computer Science, Information Technology, Management Information Systems (MIS), Data Science or related field 2+ years of experience in Apache Spark (PySpark / Spark SQL) 2+ years of experience in Python Experience with large scale streaming platforms (e.g. Kafka, Event Hub, Databricks Live Table / Structured Streaming), processing frameworks (e.g. Spark, Databricks) and storage engines stream analytics 3+ years of experience in data engineering, data integration, data modeling, data architecture, and ETL/ELT processes to provide quality data and analytics solutions 3+ years of experience in SQL with designing complex data schemas and query performance optimization Highly proficient working in Azure cloud environment (e.g. Blob / ADLS, Databricks, Azure Data Factory, Event Hub) Excellent communication skills ability to communicate technical concepts to both technical and non-technical audiences Experience in Regular Expression, CI/CD technology, Terraform and Git Keywords: continuous integration continuous deployment microsoft |
| [email protected] View all |
| Thu Sep 14 23:44:00 UTC 2023 |