| Need of Big Data Developer at Houston, TX onsite at Houston, Texas, USA |
| Email: [email protected] |
| From: lokesh, siri info solutions [email protected] Reply to: [email protected] Hello , Hope you are doing well!! Greeting from Siri Info Solutions. I am LOKESH with Siri Info Solutions. We Siri Info Solutions is a global staff augmentation firm providing a wide-range of talent on-demand and total workforce solutions. We have an immediate opening for the below position with one of our premium clients. Job Title: Big Data Developer Experience Required: 3-5 years Location: Houston, TX (Onsite) Job Summary: As a Big Data Developer within our technology team, you'll be instrumental in building, maintaining, and optimizing big data platforms and solutions. Leveraging the Cloudera suite, you will craft robust, scalable data processing pipelines and contribute to the development of our data-centric applications using cutting-edge technologies. Certifications: Cloudera Certified Professional (CCP) or any relevant Big Data certification is preferred. Key Responsibilities: Design and develop big data applications using Cloudera suite components like Kafka, HDFS, HBASE, KUDU, Zookeeper, HIVE, and Impala. Write robust, efficient, and maintainable code in Java/Scala/Python, with a strong emphasis on Spark and Flink for real-time data processing. Develop and maintain data pipelines using Apache NIFI, ensuring seamless data collection, ingestion, and distribution. Utilize Spring-Boot and Flask frameworks for creating microservices that interact with Big Data systems. Manage data workflows with Apache Oozie and orchestrate complex data processes with Apache Airflow. Implement solutions for data security and governance using Atlas, Ranger, RangerKMS, and KTS within Cloudera ecosystems. Work on cloud-native Big Data technologies, integrating solutions with cloud services for enhanced scalability and performance. Scripting and automation of routine tasks within Linux environments to enhance development and deployment processes. Optimize data retrieval with advanced HQL queries and tune performance for HIVE and Impala databases. Employ Kubernetes container orchestration for deployment, scaling, and operations of application containers across clusters of hosts. Ensure the development of high-quality applications by writing test cases and maintaining a continuous integration and deployment pipeline. Contribute to the architectural decisions and create documentation outlining design and technical specifications. Maintain a proactive approach to troubleshoot and resolve issues in the production environment. Engage with cross-functional teams to translate business requirements into technical implementations. Stay current with industry trends and evaluate new technologies for adoption into existing or new data infrastructure components. Run Data Engineering Jobs on GPUs using SPARK, tweak Jobs to utilize distributed GPUs. Design and implement security measures in the application development process, leveraging Kerberos authentication to secure communication within the cluster. Work closely with data administrators to ensure that all applications comply with established security protocols and access control measures. Develop scripts and automation tools to streamline the security aspects of the big data applications lifecycle. Implement and maintain the encryption standards for securing data-in-transit using TLS protocols to prevent eavesdropping, tampering, and forgery. Architect solutions that encrypt data-at-rest to safeguard sensitive information using industry-standard encryption algorithms and manage encryption keys with a focus on maintaining performance while enforcing data security. Regularly review and audit the code-base for compliance with data protection regulations and organizational policies. Technical Qualifications: Advanced programming skills in Java/Scala/Python with a focus on Spark and Flink for large-scale data processing. Strong understanding of the Cloudera suite, including in-depth knowledge of data management and processing services. Proficient in building applications with Spring-Boot and Flask, with experience in creating RESTful services. Experience with scripting and automation in a Linux environment, along with expertise in shell scripting. In-depth knowledge of SQL, HQL, and the ability to perform query optimization on big data sets. Proficient in Kubernetes, including deploying applications in OpenShift or other Kubernetes environments. Familiarity with Neo4j or similar graph database technologies, and the ability to integrate them into big data solutions. Experience with Cloudera Data Services such as Cloudera Data Engineering, Cloudera Data Warehouse, and Cloudera Machine Learning is highly desirable. Experience in RAPIDS & GPU-Aware Scheduling. Knowledge of data modeling, data access, and data storage techniques for big data environments. Proficient in integrating Kerberos authentication within big data applications for secure access and communication with big data services. Capable of scripting and automation to manage Kerberos ticket lifecycles, renewals, and troubleshooting common Kerberos issues in a development environment. Skilled in implementing and managing security protocols for data in transit, including setting up and configuring TLS for secure data transfer within big data solutions. Knowledgeable in encryption standards and tools for encrypting data at rest, and understanding of key management systems and cryptographic practices to ensure data privacy and regulatory compliance. Education: Bachelors degree in Computer Science, Information Technology, or a related field. Soft Skills: Strong analytical and problem-solving skills. Excellent verbal and written communication abilities. Collaborative team player with an ability to work in dynamic environments. Self-motivated with a keen interest in technology and continuous learning. Keywords: access management Texas |
| [email protected] View all |
| Fri Dec 01 01:32:00 UTC 2023 |