TATA Steel Uses Google Cloud Storage and BigQuery for Better Data Discovery and Insight Mining

Searce helped Tata Steel to move and structure their data from On-prem to Google Cloud Storage and to BigQuery, hence, creating a viable Data Warehouse.

 

Challenges
  • TATA Steel Group consisted of smaller business units and were generating a lot of data from various sources like manufacturing machines, SCADA, ERP Systems, LMS, etc. The data generated by smaller business units in TATA Steel was lying on-premise in non-redundant storage. In order to maintain compliance requirements, legacy data was stored for prolonged periods, thereby increasing the overall storage cost significantly and procuring extra storage disks in advance.
  • The humongous amount of data generated was sitting in silos in different storage systems and varied formats, so a very limited value could be extracted from this data to make informed business decisions.
  • There was no single pane of glass for the business leaders to view and analyze the entire business metrics.

To address the above challenges TATA Steel selected Searce to build an architecture supporting near real-time pipelines from all the data sources being pushed to a Data Lake after going through real-time and batch extraction, followed by the required transformations.

Searce Solution

Searce suggested TATA Steel move their organizational data from an on-premise setup to centralized cloud storage (GCS) which could handle all the raw format - the structure and unstructured data at scale. The GCS is a highly available, durable, and scalable object store, backed by SLA and having more than 99.95% uptime commitment. GCS also has a consumption-based billing that helped TATA Steel offload their legacy data into Data Lake and retire their extra provisioned storage.

Searce also helped in deploying ETL process which can push the data in an aggregated and structured format from on-prem to GCS and then BigQuery to create a data warehouse. Various batch or real-time pipelines were set up to ingest the existing and incremental data from different sources. Using the metadata features of GCS, data coming in was categorized, versioned, indexed, and processed. This also gave flexibility to business analysts to create custom reports by leveraging different data sources.

Searce also does the heavy lifting for supporting the infrastructure and Data Lake operations. As our Premium Managed Services customer, we provide 24x7 support for infrastructure to cover functions like Application Performance Monitoring, Application Maintenance, Incident Management, Change and Deployment Management.

"Data is the most valuable resource for modern business and Searce has been a pivotal partner in executing our initiatives under a digital strategy. By implementing Data Lake, we have witnessed tangible benefits globally across our production, supply chain and customer service units. We can now plan better, predict faster and act in an informed way."

Gaurav Jalan, IT Infrastructure Manager, Tata Steel

Business Impact
  • Business overview available through a single pane of glass. Leaders were able to extract more value out of the same existing data, capture data and correlate activities and map events across all the systems and machinery based out of 26 different plants and 100+ sales offices, visualize them and make informed decisions.
  • Improved Supply Chain - Enhancing the overall customer experience by planning, executing and monitoring last-mile deliveries and order fulfilment. Removing bottlenecks in the process and ensuring customer commitments are met.
  • Better portfolio mix and increased profitability - Creating an optimized product mix to create long term value to customers and generate repeat business.
  • Increased production planning accuracy - Optimize demand-supply balance by forecasting future demand and seasonality and regulating production capacity, so TATA Steel was able to save 17 Million USD annually.
  • Agile deviation management - Using predictive analytics and granular monitoring, errors and deviations are now highlighted well in advance and management is now able to mitigate these issues too.
  • Exploration and Analytics - Using Data Lake, TATA Steel was able to open avenues to new possibilities. Data scientists could create ML models, collaborate their work and train models to drive future tech changes which are likely to boost growth and sales. Data residing in Data Lake supports metadata, cataloguing for easier discoverability and exploration of data by data scientists.
  • Lower downtime and shorter maintenance windows - TATA Steel used predictive analytics to analyze the upcoming bottlenecks and plan maintenance accordingly, thereby reducing the downtime and increasing productivity.
About Tata Steel
Tata Steel

TATA Steel Group is among the top global steel companies with an annual crude steel capacity of 34 million tonnes per annum (MTPA). They are one of the world's most geographically diversified steel producers with plants in 26 countries and serving clients in 100+ countries.

Industry: Mining and Metals
Location: India