Home

Pavan Kumar - ETL Developer

[email protected]

Location: Remote, Remote, USA

Relocation: Yes

Visa: TN

PROFESSIONAL SUMMARY ETL developer with over 13 years of experience in complete software development life cycle with design, development, integration, maintenance, enhancements, debugging/ troubleshooting, testing and production support. Experienced in working with Standard jobs, Batch jobs, Streaming jobs in Talend Open Source, Talend Enterprise Version and Talend Cloud. Expertise in Data cleansing, data modeling, ETL and data analysis of the Data Pipeline. Design and development of ETL applications with efficient mappings for multiple sources and targets. Design and development of ESB integration applications, REST API web services based on RAML and JSON. Transformation of data using expressions and Java programming in Talend. Well versed with Performance tuning, Error handling and debugging the applications using Java Debug and Traces Debug of Talend. Expertise in developing applications, ETL processing, big data integration using Talend Real time big data for Apache Spark and Apache Hadoop Big Data. Has extensively worked on Talend open studio for Big Data, Cassandra, MongoDB, AWS S3, AWS Kinesis Streams. Handled very huge and very fast transaction data of the IoT sensors in Real Time and Batch processing. Extensive experience in developing applications for HDFS, Kinesis & S3, SQL, NoSQL, web services, flat files. Integration and migration of the data from legacy on-premises MySQL databases to the cloudbased Snowflake Extracted Data from multiple sources and implemented SCDs (Type 1/ Type 2/ Type 3) using Talend. Worked with different input and output files like CSV, TEXT, JSON, XML. Deploying and scheduling of applications in the cloud using Talend Management Console and runtime remote engines. Configuring and maintaining NoSQL Cassandra Big Data Server for high availability using the Clustering technology. Talend Certified Developer TECHNICAL SKILLS Data pipeline aggregation queries in MongoDB. Creation of effective Test data and development of thorough Unit test cases to analyze bugs and ensure successful execution of the data. Worked in Agile, Scrum environments with Jira and Git for versioning & repository Excellent Client interaction/communication skills and proven experience in planning, scheduling, and coordinating all aspects of projects. Integration ETL, Integration, ESB, REST API webservices Big Data Apache Spark, Hadoop HDFS, AWS S3, AWS Kinesis, Cassandra NoSQL, MongoDB Testing & Bug Tracking Jira, Bugzilla, Postman, SOAP UI, Compass Tools/IDEs Talend Real Time Big Data Studio Build System Maven Methodologies Agile, Scrum Versioning Control Git, Bitbucket Object Oriented Programming C++, Java Database SQL, NoSQL Professional Experience Tata Consultancy Services Canada Inc October 2020 till date Project: High Value Payments Role: ETL Developer Responsibilities: Developed the Talend jobs for Extracting, Transforming, Loading (ETL) the data into NoSQL MongoDB database. Designed and developed the jobs for handling high volumes of data (more than 1 Million records per day). Developed the jobs for the reconciliation of the data. Developed the jobs for generating the reports from MongoDB data. Developed aggregation pipeline queries in MongoDB. Developed Java routines for transforming the data. Developed the ETL jobs for Batch processing and Real time processing. Ingestion of data into MongoDB database from REST API, IBM MQ, NDM and SFTP. Developed the ETL job for extracting the data from different file formats (CSV, JSON, XML, fixed length position, multi-schema). Worked with nested objects and embedded arrays in JSON structure. Handled different error cases with Audit logs. Deployed the ETL jobs in the DEV, UAT, PROD environments. Talend Certified Developer Used Jira for Agile Scrum methodology. Used BitBucket for code versioning and repository. Merging of feature branches into develop, master and release tag branches. Unit Testing, debugging and error handling of ETL jobs. Used Git and Jira for raising the PULL REQUEST for code review and merging in Git. Participated in the Scrum meetings for Program Increment planning, sprint planning and daily standup meetings. Spreems Inc April 2020 October 2020 Project: Web-based Integration App Role: Integration Developer Responsibilities: Developed the jobs for the REST API and for Extracting, Transforming, Loading (ETL) the data into NoSQL Big Data Clustering database. Designed the RAML file for the REST API. Deployed the REST API and ETL jobs in the Azure Cloud. Data modeling for NoSQL Cassandra database. Jira for Agile Scrum methodology Unit Testing, debugging and error handling of ETL and REST API jobs. Used Git and Jira for raising the PULL REQUEST for code review and merging in Git. Participated in the Scrum meetings for sprint planning and daily standup meetings. Spreems Inc October 2019 March 2020 Project: Trivedi Advanced Technologies LLC Role: Integration Developer Responsibilities: Developed ETL Jobs for streaming the IOT device s data from AWS Kinesis streams and loading to Cassandra database. Transforming the Kinesis streams data into JSON format Extracting the data from NoSQL database and display in JSON format Designed the RAML file for the REST API. Developed the jobs for REST API. Deployed the REST API to AWS cloud. Worked on AWS S3, AWS Kinesis Streams, Spark Streams, SQL, NoSQL components connectors for Real time Processing Worked on Hadoop HDFS connectors for batch processing Worked on flat files and XML components in Talend Data modelling for NoSQL Cassandra database. Unit testing of ETL jobs and debugging, error handling. Created Postman Test suite for testing the REST API jobs Bug fixing, raising PULL REQUEST and merging the code for Jira epics and stories after QA. Participated in scrum meetings with various teams to coordinate project developments and issues. Talend Certified Developer Spreems Inc March 2019 September 2019 Project: Air Canada Role: Integration Developer Responsibilities: Designed the RAML file for the REST APIs. Developed the REST API applications in the system, process, and experience layers Developed REST APIs using REST HTTP connectors and deployed to the cloud. Used the Proxy gateway and applied security policies - SLA tiers. Extensively Transformed data of different formats like XML, JSON, CSV. Using Jira for Agile Scrum methodology Conducted unit tests of the application and testing of REST services using postman. Involved in Unit Testing of REST APIs using Postman by performing GET, POST, PUT and DELETE actions. PULL REQUEST for code review and merging with Git. Participated in the everyday Scrum meetings. Dover Corporation May 2017 April 2018 Role: Senior Software Engineer Responsibilities: Developed the Data Integration for IOT device with Cloud. Developed the application in the system, process, and experience layers Developed REST APIs using HTTP connectors and deployed to the Cloud. Transformation of the data from different components of different formats. Developed the flows to collect IOT device data and send to HTTP connectors and FTP connectors Used the CSV flat files connector in the flows. Tested the complete flow of data from IOT to Cloud. Bug fixing for epics and stories after QA and worked error handling strategies Used the Bitbucket for code versioning source control and Jira for Agile Scrum methodology Participated in the daily standup Scrum meetings and Involved in design discussions. Developed various test cases for unit testing and Supported the testing team for integration testing Dover Corporation May 2013 April 2017 Role: Software Engineer Responsibilities: Developed embedded web server using TCP/IP for configuring the Cloud Worked on Saving and retrieving of configuration data from MySQL database Developed the Html pages for displaying the extracted data from MySQL Developed the interface for executing the MySQL queries and displaying in web using HTTP protocol in MVC architecture Developed interface for the Point of Sale (POS) protocols using TCP/IP Talend Certified Developer Worked for continuous logging of system report information in MySQL database Developed GUI screens and touch buttons, alert messages for different tabs of different screen pages for the LCD touch and display screen Handled the integration of data between the threads in the system Handled the synchronization of the complete system having more than 40 continuous loop running threads Global Edge January 2011 February 2013 Role: Software Engineer Responsibilities: Solved technical issues including bug-fixing of GUI of the embedded device with Object Oriented Programming Troubleshooting and resolving the bugs of the Network Audio Player (NAP) layer using the Wireshark. Worked towards resolving of networking issues. Modified the GUI (MVC architecture) for the embedded device in C++ Hewlett Packard (HP) March 2008 March 2010 Role: Consultant Responsibilities: Development of the prototype of the 3D gesture recognition Developed C++ object-oriented code for capturing of raw data from accelerometers and transferring it to PC (USB to serial communication). Developed the MATLAB code for Processing the accelerometers data and displaying the gesture characters on the PC screen Keywords: cplusplus quality analyst user interface message queue sthree information technology hewlett packard