| Pyspark / Big Data Architect : Charlotte NC (Onsite) : CGEMJP00271528 at Charlotte, North Carolina, USA |
| Email: [email protected] |
| Hi Team, Hope you are doing well. Request you to please share the profiles for the below role to [email protected]. Role : Pyspark Architect Location :Charlotte NC (Onsite) Duration : 12 Months Job Description : The ideal candidate will have 10+ years of experience in software architecture, data engineering, and large-scale data processing systems, with a strong focus on PySpark. Experience in Finance Technology or Enterprise Function technology domains will be a significant advantage. This role requires a leader with a strategic mindset who can design, implement, and oversee high-performance, distributed data processing systems. Key Responsibilities: Lead the architecture, design, and implementation of large-scale distributed data systems using PySpark. Collaborate with business stakeholders, technology teams, and data engineers to gather requirements, define objectives, and build scalable data pipelines. Drive end-to-end solution design, including data acquisition, storage, processing, and analysis. Optimize performance of big data processing systems, ensuring low-latency and high-throughput data flows. Ensure alignment with industry best practices and compliance standards in data security and privacy. Mentor and guide a team of developers and engineers, promoting best practices in coding, architecture, and design patterns. Evaluate new tools and technologies, identifying opportunities for innovation and driving their implementation. Collaborate closely with cross-functional teams, including finance and enterprise functions, to ensure solutions meet business objectives. Support critical decision-making and roadmapping to enhance the organizations data processing capabilities. Qualifications: 10+ years of experience in software architecture, with a strong focus on PySpark and big data processing systems. Proficient in Apache Spark, Hadoop, and other distributed computing frameworks. Deep understanding of data architecture, ETL/ELT processes, and cloud-based data platforms. Proven experience in Finance Technology or Enterprise Functions is highly desirable. Strong knowledge of relational databases, NoSQL databases, and data warehousing solutions. Solid experience in working with cloud platforms such as AWS, Azure, or Google Cloud. Hands-on experience with streaming technologies like Kafka and real-time data processing. Excellent problem-solving skills, strategic thinking, and a results-oriented approach. Proven leadership abilities, with experience managing technical teams in a fast-paced environment. Strong communication skills, capable of presenting ideas clearly to both technical and non-technical stakeholders. Preferred Skills: Experience with financial systems or enterprise applications. Familiarity with machine learning frameworks and AI-driven data insights. Experience with DevOps practices and CI/CD pipelines in data engineering." -- Keywords: continuous integration continuous deployment artificial intelligence information technology North Carolina Pyspark / Big Data Architect : Charlotte NC (Onsite) : CGEMJP00271528 [email protected] |
| [email protected] View all |
| Thu Oct 17 19:53:00 UTC 2024 |