Hitachi India Pvt and the Indian Statistical Institute on Monday announced the joint development of HISIA, a data analysis method that identifies the type and amount of historic data effective for accurate forecasting. The Hitachi and ISI Analysis (HISA) provides a theoretical assessment to judge the amount of data that can minimise the estimation error for a given confidence level, said an official media release.

In the case of heterogeneous data, it is hard to make preventive diagnostics and demand forecasts. For example, prediction accuracy may not be significantly improved by increasing the database for an electricity demand forecasting model based on the maximum and minimum temperature of historic data from the past several decades. This could be because the preconditions that influence demand, such as lifestyle, electricity tariff and energy-efficiencies of equipment, may have changed.

The R&D Centre of Hitachi India and the ISI Bangalore, investigated this issue to determine the type and amount of data necessary to achieve accurate forecasting, and minimise time and money spent on collecting data.

The team developed HISIA, which performs a theoretical check on the structural changes in historic data and quantifies the error in estimation, both before and after the change.

This assists data scientists to determine the limitations in improving the accuracy of estimates and optimising the granularity of clustering. The structural changes in the historic data are like the “tipping point” beyond which more data does not significantly increase accuracy but instead contributes to “noise” due to the heterogeneity.

The HISIA model was then applied to the case of electric load forecasting. For this case study, it was found that the electric load for a New York utility at 8:00 AM could be estimated with minimal quantified mean square error using just three years of historic load data.