Migrate Enterprise DWH to Scalable and Cost-Effective Modern Data platform using Azure Synapse
With a 75-year track record, Pike Corporation is a leading provider of turnkey infrastructure solutions for electric and gas utilities, as well as telecommunications companies. On the electric side, Pike specializes in the engineering, construction, maintenance and upgrade of transmission, distribution and substation infrastructure. Pike’s expertise ranges from traditional and renewable grid assets and storage, to longer-range smart city planning and implementation. Pike is also a leader in emergency storm restoration. Pike provides similar turnkey engineering and construction solutions for telecommunications clients, supporting the full range of wireline and wireless infrastructure projects, as well as for gas utility clients. The company is taking full advantage of the Azure Synapse Analytics to migrate existing Enterprise Data Platform to Scalable, Elastic and Cost-Effective Modern data platform. Since Pike used Microsoft Dynamics, Synapse provides direct link to synchronize the ERP and CRM data into Synapse Warehouse without much additional effort.
Utility (Telecom, Power, & Engineering)
Location: North America
Rising customer expectations around times promise
Lack of technician knowledge
Not an easy solution to support during breakdown
Unable to Scale due to existing tools inability
Synapse: This is a complete BI solution which provide built links to many azure components like ERP, Azure SQL, Power BI etc. This will help client to easily sync the transaction data to Data warehouse data. This component has replaced Azure Data Factory, Azure Functions and Data Bricks. This is used for triggering the Oracle BICC job, download and unzip files and also for ETL. Since Synapse is having Spark component which is highly scalable and flexible to process the data at a fast pace
Delta Lake: Delta lake is data warehouse built on top of data lake. The databases Bronze, Silver and Gold are created in data lake which is high cost effective and it can store large amounts of data
Synapse Serverless Pool: This is a On-Demand pool for accessing the delta tables. No storage will be created except that it provides SQL end point for connection. Money will be charged only for the amount of data queried.
Power BI: This is an Enterprise reporting tool used by the client, existing dashboards are migrated with minimal changes to in the connections to Synapse On-Demand Pool
Lower Cost - Once the data is loaded into Delta-lake we will use Serverless SQL pool. With server less pool we will just pay price based on the size of data we are trying to extract.
Auto Scalability - Since Synapse notebooks use Spark Clusters ETL pipelines can scale upward and outward to meet the computing, processing and storage requirements. This can be achieved with spark cluster easily. Since it is elastic, it can easily scale up or down based on the need.
Flexibility - Since Synapse notebooks support coding, we have migrated the code from Azure Functions and Databricks into Synapse notebooks which can be used for downloading the data from Oracle PVO. All these things can be achieved in Synapse notebooks itself.
Schema Evolution - In delta it is easy for us to enforce schema evolution as it is not easy to do it in Azure SQL DB and ADF pipelines.