Home

Adithya Pemmasani - Azure data engineer

[email protected]

Location: Dayton, Ohio, USA

Relocation: yes

Visa:

Adithya Pemmasani Email:[email protected] Phone no: (936) 410-4777 Summary: 7 years of IT experience with Microsoft and Cloud Technologies (Azure) as Database Developer, Data Engineer, BI Developer providing data services in design, development, extracting, processing, loading, data mining, analyzing, reporting, visualizing and business intelligence solutions using Azure Data Lake Storage, Azure Data Factory, Microsoft Flow, Azure Storage, SQL Server. Experience in building big data solutions using Azure services like Azure analysis services, Azure Data Warehouse, Azure Databases like SQL server, and data orchestration in ADF. Experience in data-related design tasks that include designing Azure data storage solutions that use relational and non-relational data stores, batch and real-time data processing solutions, and data security and compliance solutions. Realtime experience in using Azure services: Portal, Azure Synapse, Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Azure Blob storage. Design, develop, maintain and monitor data pipelines using Azure Data Factory for real-time/batch analysis, reporting, optimization, data collection, and related functions. Developing and maintaining a CI/CD pipeline that manages and deploys data pipelines, data infrastructure and data artifacts as code in Azure DevOps. Migrate existing data pipelines and data from On-Premises SQL Server and other environments to Azure Data Lake and Azure Blob Storage using Azure Data Factory. Experience in OLTP & OLAP Data warehousing applications Interpreting and working with data models, data flows and data dictionaries, logical & physical Data modeling, data loading in Azure Data Warehouse (Azure Synapse). Good knowledge of Data Marts, Data warehousing, Operational Data Store (ODS), OLAP, Data Modeling like Dimensional Data Modeling, Star Schema Modeling, Snow-Flake Modeling, FACT and Dimensions Tables using Azure Analysis Services. Design and implement data load processes from On Premises sources into Azure Data Lake and subsequent Azure SQL & SQL Data Warehouse using AZCopy, Azure Data Factory, Azure .NET libraries. Developing OLAP cubes using SSAS 2005/2008R2 and Expertise skills in calculating measures and dimension members in SQL Server Analysis Services (SSAS 2008R2) using multi-dimensional expression (MDX), mathematical formulas, and user-defined functions. Relational Database Management Systems (RDBMS), Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), and other relevant technologies. Involved in large data migrations, transfers using utilities like Data Transformation Services (DTS), SSIS, Bulk Copy Program (BCP), Bulk Insert, Azure data factory and Azure Data bricks. Highly experienced in whole cycle of DTS/SQL server integration services (SSIS 2005/2008R2) Packages (Developing, Deploying, Scheduling, Troubleshooting and monitoring) for performing Data transfers and ETL Purposes across different servers. Experience in providing Logging, Error handling by using Event Handler, and Custom Logging for SSIS Packages. Extensive database programming Experience in writing T-SQL Stored procedures, User Defined Data Types, Triggers, Constraints and Indexes using various DDL and DML commands, CTEs, SQL functions. Using all kinds of SQL Server Constraints (Primary Keys, Foreign Keys, Defaults, and Check, Unique etc.) and Generating Complex Transact SQL (T-SQL) Queries, Sub queries, Co-related sub queries, Dynamic SQL queries etc. Tuning the performance of Stored Procedures and large T-SQL Queries using clustered and non-clustered Indexes, DMV s and efficient coding standards. Writing T-SQL queries (DDL and DML). Creating SQL Scripts for data loads and upgradesfor data migrations and data validations. Involved in Installation, Configuration, Development, Deployment and Troubleshoot of MS SQL services in different environments (PROD, PPE, UAT, TEST, DEV) and using other tools like SQL Profiler, Tuning Advisor, SSMS, SSMA. Develop, publish and maintain data reports, visualizations, and interactive dashboards for internal and external customers using Microsoft Power BI, Power apps, Power-BI Desktop and other business analytics tools. Experience in writing complex calculated columns, measures and functions using DAX. Integrated custom visuals based on data requirements using Power BI desktop. Expert in creating Power pivots, Power views and SSRS reports using tabular model (DAX), Cubes (MDX) and SQL queries. Experienced in creating reports in drill mode using tables, financial tables, cross tabs and charts using SSRS. Skills & Tools Technologies / Languages SQL, T-SQL, Python Cloud Technologies Azure Data Warehouse (Azure Synapse Analytics) Azure Data Lake Storage, Azure Blob Storage Azure SQL Database, Azure SQL Server Azure Data Bricks, Azure Data Factory Azure Data Explorer, Azure DevOps, Azure Portal Databases MS SQL Server 2008-2018 ETL Tools Azure Data Factory, SSIS (SQL Server Integration Services) Business Iintelligence/Reporting Power-BI reports Other Tools/Technologies SQL Server Management Studio (SSMS), SQL Profiler, SQL Server Migration Assistant (SSMA), Excel, Azure, Git Education Chaitanya Bharathi Institute of Technology Jun 2011 - May 2015 Bachelor of Arts - BA, Computer and Information Sciences and Support Services Grade: 91% Certifications SQL Fundamentals - SOLO LEARN Big Data Foundation-IBM Cloud Experience Azure Data Engineer | Client: SUNRUN | June 2021 - present Denver CO Responsibilities Create a Datawarehouse architecture for end users to consume data seamlessly. Ingest data into data lake using Azure data factory, Azure Databricks and python notebooks using python and SQL code. Design and create hive Meta store tables, partition, archive, and purge tables for efficient load strategies. Design and architecture ADLS data in Azure Synapse to expose as external tables and internal tables. Create Databricks delta tables, Hive tables & photon clusters for optimal performance. Create high concurrency interactive cluster & mount ADLS data store containers. Create Azure data bricks job clusters for data ingestion and create python notebooks. Designed and implemented a fault-tolerant batch processing architecture using Azure Databricks, Azure Data Lake Storage, and Azure Batch, processing large datasets in parallel. Developed complex ETL (Extract, Transform, Load) workflows in Azure Databricks, combining data from multiple sources, performing aggregations, and generating summary reports. Utilized Azure Databricks Delta Lake for managing batch data storage, enabling efficient schema evolution, versioning, and data consistency. Design data pipelines to orchestrate data by using data factory and data bricks loads for reporting and semantic tables. Enable logic apps and configure emails so the users are updated over the load success and failures. Maintain over 1000 data table's schema and data, that include 500+ ingestion, 200+ stage tables and 180+ reporting tables etc. Administer multiple ADF workspaces, for each set of users with restricted data accessibility. Monitor the Azure ADLS Gen2 environment and test tableau reports with synapse data loads. Create service tickets to Microsoft team, follow-up on Azure platform performance and infra structure related issues Azure Data Engineer | Client: STARBUCKS | July 2020 June 2021 Seattle, WA I am working as Data Engineer/ BI Developer providing data services (data engineering, data mining, ETL, reporting, visualization) on raw data, customer data, product growths, quality of services, KPIs and metrics. Responsibilities Developed and maintained data models in Azure Databricks Delta Lake, ensuring data consistency, reliability, and ease of use for downstream applications. Assisted in migrating on-premises data infrastructure to Azure cloud, including setting up Azure Databricks clusters and migrating existing data workflows. Conducted data analysis using Azure Databricks notebooks, performing exploratory data analysis, and generating actionable insights for business stakeholders. Contributed to the development of a data catalog using Azure Databricks and Azure Data Catalog, enabling easy discovery and access to data assets. Developed SCOPE, C#, U-SQL scripts for data mining and generating data streams for Azure Active Directory Services. Worked on COSMOS Big Data by running U-SQL queries with token allocation, bonus tokens, submitting jobs to virtual cluster, views, row sets, input, and output streams for Structured and Unstructured data. Designed, created, and monitoring data pipelines to extract data from Azure Blob Storage, Azure Data Lake Storage, Azure Cosmos DB using Azure Data Factory and injecting into Azure SQL Data Warehouse. Comprehensive Azure database management and migration of millions of user's data from Cosmos (Microsoft Big Data) and App Insights to Azure SQL and Azure SQL Datawarehouse. Developed and designed of tables, SQL Queries, CTE Queries, functions, store procedures, schemas, scripts, SQL Jobs, views, SCOPE Script, Kusto, iScope Scripts in SQL Data Warehouse and Cosmos DB. Building and writing scripts for data modeling, mining to give easier access to Azure Logs, App Insights to PMs & EMs. Worked on creating data sets, linked connections, data activities, triggers, and alerts in ADF for ETL operations. Created Azure Key Vaults for securing credentials and service principals to connect Azure resources with secret keys. Creating data workflows and scheduling on daily/weekly to feed dashboards with latest data to ensure how released Azure products are working. Creating reports and dashboards in power-BI desktop and publishing to power bi workspace, scheduled data refresh on daily basis, configure alerts for anomaly activities in power-bi server. Working on data analysis and reporting (using Power BI) on customer usage metrics. I used this analysis to present to the leadership towards a product growth to motivated team of engineers and product managers. Delivering monthly and quarterly detailed reports to leadership teams and across the board on product growth, QoS, newly onboarded customers and key activities on customer insights. Create technical design documents for new features to be developed by the team with detailed requirements evolved during discussions with Product Owners and Engineering team. Automate the data workflows by Microsoft Flow and ADF to feed the Power BI dashboards. Create automate IcM tickets for missing data catalogs assigning to respective team, monitor and backfill the data. Handle the requests for SQL objects, schedule, business logic changes and Ad-hoc queries from customer and analyzing and resolving data sync issues. Planning for releases of new developed modules, manage schedule and quality of each deliverable for client applications and tools. Technology Skills: Azure data factory, data lake storage, blob storage, tables, Azure devops, sql server, SQL data warehouse, Microsoft flows, power-bi, python, Kusto db, t-sql, data bricks, ci/cd pipelines, azure logs, azure portal, dax queries, key vaults, Git. Sr Database Developer/BI Developer | Client: CREE INC| Apr 2017 June 2020 Worked as Sr Database Developer/BI Developer for client Cree, migrated databases with schema and data from SQL 2000 servers to SQL 2014, redesigned database objects (tables, views, procedures, functions, queries) and upgrade servers by resolving version conflicts. Responsibilities Worked on Installation, Configuration, Development of MS SQL services in different environments (PPE, TEST, DEV) with different versions 2000, 2008, 2014 using other tools like SQL Profiler, Tuning Advisor, SSMS, SSMA. Created tables, views, complex queries, SQL joins, stored procedures, functions, UDF s, CTE, triggers, tracing, and performance tuning queries for migration objects from SQL Server 2000 version to 2014 version. Worked on resolving SQL code conflicts in SQL Server 2000, 2005, 2008, 2014 versions. Migrated SQL objects and data as well by writing SQL queries, bulk copy and designing SSIS packages for data load. Responsible for DTS and SSIS package migration for Visual studio 2000, 2008, 2013 versions. Identified deprecated functionality in 2005 from 2000, and recreated SQL scripts for objects in databases and servers. Worked on VB script or C# script modification in script task components in DTS and SSIS ETL packages. Recreated DTS packages functionality in SSIS ETL packages and resolved ODBC connection issues in SSIS packages. Written code in SQL to resolve RAISEERROR, TEXTPTR, UPDATETEXT syntax issues and removed text, ntext, varbinary datatypes and re-create with appropriate datatypes in SQL Server 2008, 20012, 2014 versions. Documented well SQL migration code snippets for client reference and future deployments or PROD data migration. In migration phase-1 & 2 completed 46 DTS, 226 SSIS packages, 584 servers with 1909 databases. Technology Skills: MS SQL Server 2000/2005/2008/2014, SSIS, SQL Profiler, TFS, Visual Studio, SSMS, Microsoft migration tool (MMU), Script Analyzer, DTS Analyzer, SQL Server Migration Assistant (SSMA), T-SQL, SQL. Database Developer | Client: ATHENE| Aug 2015 Mar 2017 West Des Moines, IA I worked as Sr Database Developer & DBA, responsible for SQL server database application design, development, maintenance, monitoring, deployments, administration, SSIS packages, Query tuning, and backup management. Created migration plan for existing platform databases as schema first next data from SQL Server 2005, 2008 to 2012 without losing customer s data (Orders, products, shipping, customers, store settings, admin configs). Responsibilities Installed, configured, data backup and security settings for standalone SQL Server 2008r2, 2012, 2014 Server. Designed, created tables, views, complex queries, SQL joins, stored procedures, functions, UDF s, CTE, triggers, tracing, SQL logging, sub queries, index management (clustered, non-clustered) in DEV, PPE, PROD servers. Worked on SQL Server Constraints (Primary Keys, Foreign Keys, Defaults, and Check, Unique etc.) and created T-SQL queries (DDL and DML), Sub queries, Co-related sub queries, Dynamic SQL queries etc. Design and crested SSIS-ETL packages for data load using transformations like For Each Loop, Execute Package/SQL tasks, Script task, send mail task in Control flow and Conditional Split, Data Conversion, Derived Column, Lookup, Fuzzy lookup, Aggregation, etc. Created SSIS-ETL packages to validate, extract, transform and load data into data warehouse, databases using data conversion transformations and implemented event handlers and error logging, failure alerts. Maintained the physical database by monitoring performance, integrity and optimize SQL queries for maximum efficiency using SQL Profiler and Extended Events, Database Tuning Advisor, and Index Tuning Wizard. Worked on DBA activities by database Integrity checks, update DB statistics and re-building and re-creating indexes. Created maintenance plans and backup jobs in SQL Agent for efficient recovery by setting up full/differential backup and more frequent transaction log backups for DEV, PPE, PROD SQL server s databases. Created automated deployment workflows on database changes in DEV, UAT, PPE, PROD environments. Proactively worked on performance, dead lock issues, data issues, missing indexes, disk and database file issues. Documented standard and maintain SQL Server documentation including DBA manuals for every server including design, installation, and configuration, as well as detailed work logs. Worked on installation, administration, monitor, security, performance, troubleshoot, query tuning and other database admin Activities in PROD, PPE, TEST, DEV environments. Worked on design, develop, and modeled large scale databases and implemented SQL programming for application flow using database objects like tables, views, functions, store procedures, DDL, DML SQL queries, and CTE queries. Redesigned on existing application database with extensive knowledge of Data mapping, Data Modeling, Snowflake, Star Schemas, SSIS ETL jobs and OLAP operations, data model for new application. Creating DDL, DML queries, SQL joins, SQL stored procedures, views, UDF s, tracing and performance tuning for eCommerce product development, PPE testing and PROD deployments. Designed and implemented database maintenance plans covers update statistics, index rebuild/recreate and resolving deadlocks to improve database performance. Created complex SQL queries to build e-Commerce reports and dashboards on product sales, revenue metrics, order lifecycle and shipping management for retailer s and supplier s business growth. Created SSIS Packages using Pivot Transformation, Fuzzy Lookup, Derived Columns, Condition Split, Term extraction, Aggregate, Execute SQL Task, Data Flow Task, and Execute Package Task, etc to generate underlying data for the reports and to export cleaned data from Excel Spreadsheets, csv file, text file to database. Experienced in creating multiple reports in drill mode using tables, financial tables, cross tabs and charts using SSRS. Technology Skills: MS SQL Server 2008/2008R2/2012/2014, SSIS, Tuning Advisor, Resource Governor, SQL Profiler, TFS, Visual Studio,SSMS, T-SQL, SQL Keywords: csharp continuous integration continuous deployment business analyst artificial intelligence access management business intelligence database active directory information technology microsoft Colorado Iowa Washington