| Data Engineer (SQL Replication + data bricks SME abilities)| Must be local in Dallas for onsite weekly one day each week at Dallas, Texas, USA |
| Email: [email protected] |
| Role- Data Engineer (SQL Replication + data bricks SME abilities) Location- Must be local in Dallas for onsite weekly one day each week Duration- 6-12 months Visa- No H1B, OPT, CPT Must be local to Dallas who will work onsite 1 day per week (on Tuesdays- they go in Tues and Weds and we ask our guys to go in each Tuesday). Challenge We have data processing system that is being modernized. The first part of the system is modern, using Databricks as a high speed ETL engine. The second part is legacy, using Microsoft SQL Server Replication to distribute data. The challenge is that the legacy part of the system cannot keep up with volume and velocity of the data produced by the modern part and will severely bog down, impacting our customers who rely on this data. We have had customer impacting events at least once per month since March. We know there are techniques to mitigate this but there are no experts in SQL Server or replication available internally. Additionally, we do not have a technical SME who can explain the legacy code. What we need We need an expert who has a depth of hands-on experience using Microsoft SQL Server (9+ years) and who is particularly experienced in using replication. This person would be very skilled in SQL, and able to interpret/extrapolate business logic from SQL code. Additionally, they should know end-to-end data processing techniques and have modern data processing experience using Python and/or Spark 6+yrs). Databricks(3+yrs) experience. The person should have good experience in data integration and working in cloud (Azure preferably). Success criteria At the end of the engagement, we will have a more resilient SQL Server Replication operations with a measurable reduction of customer impacting events resulting from SQL Server Replication. We will have documentation of what the legacy code does and the changes that were made to strengthen the process. This should include any technical debt incurred due to the speed of implementation we require. Thanks & Regards, Vishal Kashyap Technical Recruiter Email- [email protected] | 806, New Castle , Wilmington, DE, US, 19801 -- Keywords: information technology golang Delaware Data Engineer (SQL Replication + data bricks SME abilities)| Must be local in Dallas for onsite weekly one day each week [email protected] |
| [email protected] View all |
| Sat Aug 17 00:21:00 UTC 2024 |