| Lead/Senior Big Data Engineer (Exp with Telecom company) at Bedminster, NJ (Hybrid) - 15+ yrs Profile at Remote, Remote, USA |
| Email: [email protected] |
| Lead/Senior Big Data Engineer (Exp with Telecom company) Location - Bedminster, NJ (Hybrid) (Local or nearby only) Experience 15+ Years in relevant field Visa USC, TN, H4EAD, OPT EAD (PP is must for all except USC) Must Skills - Apache Spark, Elasticsearch, OpenShift, Hadoop, ETL, Scala, AWS Project summary: The project is in telecom sector and involves collecting, processing and analyzing large volumes of network performance data from various sources, including mobile devices, CPEs, scanners and third-party testing tools. Project features with complexity of its data sources, the volume of data processed, and the need for real-time analysis. Job Description: Bachelors degree in Computer Science, Software Engineering or related field 10+ Years of experience working on any public cloud platforms, preferably AWS and an excellent grasp of the following: o Development of Complex Data Ingestion & Transformation Pipelines o Spark Programming hands-on experience (Scala, Python, PySpark, Spark SQL) o Strong Apache Spark and Elasticsearch expertise. o Good understanding of OpenShift, specifically running Elasticsearch on OpenShift o DevOps skills, preferably ArgoCD would be a big plus o Databricks/Hive/Datalake/Deltalake Excellent grasp of data warehousing and ETL/ELT use cases and patterns at scale The ability to apply such methods to solve business problems in combination with building data pipelines, data streams, and system integration. Support the planning and implementation of data design services, providing sizing and configuration assistance and performing needs assessments. Translate business requirements to technical solutions leveraging strong business acumen. Analyze current business practices, processes, and procedures as well as identify future business opportunities for leveraging cloud services (Microsoft, Amazon, and/or Google). Deliver architectures for transformations and modernizations of enterprise data solutions using cloud data technologies. Develop and maintain data warehouse schematics, layouts, architectures, and relational/non relational databases for data access and Advanced Analytics. Implement effective metrics and monitoring processes. Demonstrated experience of turning business use cases and requirements to technical solutions. Experience in business process mapping of data and analytics solutions. Experience in leading and mentoring development teams. Ability to model business processes using industry best practices. Ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flows. Amazon AWS Experience a big plus. Job Responsibilities Design and develop scalable data processing systems using Apache Spark and ElasticSearch, Deploy and manage ElasticSearch on OpenShift for indexing and data storage. Create customer libraries for high-bandwidth data processing and ETL Develop solution for handling large data processing and transfer distributed system and storage environments Implement and maintain ETL processes to integrate large datasets into data lakes and data warehouses. Lead and mentor our development teams to deliver a quality data solution. Work closely with other app developers and engineers to ensure project success. Exercise a high degree of autonomy and ownership Research approaches to solve unfamiliar problems Warm regards, Reena Felix Recruitment Manager Email: [email protected]/ [email protected] linkedin.com/in/reena-felix-747260248 -- Keywords: information technology trade national New Jersey Tennessee Lead/Senior Big Data Engineer (Exp with Telecom company) at Bedminster, NJ (Hybrid) - 15+ yrs Profile [email protected] |
| [email protected] View all |
| Tue Oct 08 01:23:00 UTC 2024 |