Banks embarking on Enterprise Data Warehouse (EDW) projects have of late been on the increase. This is an impact of India’s primary regulator the RBI, asking Public Sector Banks to implement a Central Data Repository, which is actually another avatar of a Data Warehouse (DW). While a DW does deliver great benefits, costs have been escalating irrationally. Not so long ago, large Banks (in the 3000 to 5000 branches league) were able to implement an EDW by investing around INR 50 crores including the cost of about 5 years of post-implementation support. Project costs today for the very same exercise have gone up by a whopping 100%. Besides the cost factor, most Banks are dissatisfied with their DW.
This makes one ask ‘is it really worth spending an astronomical sum for a DW?’, and more importantly ‘What is the true value Banks derive out of a DW exercise’? The second question that naturally comes up is ‘Is it not possible to get the same (or better) benefits at a lower cost?’ To understand the state of affairs in depth, let’s first take a quick look at the very purpose of establishing an EDW.
Banks typically expect the EDW to enable –
- Regulatory reports
- Ad-hoc reports
- MIS and Dashboards
- Analysis of Business trends
- Product Gaps and opportunities
- Analysis of Delivery Channel Transactions, identifying new initiatives
- Performance Tracking v/s Budget
- Customer Analysis and Pricing
- Cross selling
- Promotional Campaigns
- Exposure and Risk Management
- Social Media Analytics
Each Bank must dispassionately examine its specific requirements and demand only required functionalities. Also, these requirements should be in phases, with each phase not exceeding six months. The DW project delivery schedule should be clearly defined. Wish lists and good-to-haves should be excluded in phase one. While Data ETL should be a combined activity, the phases of Data modelling and implementation of application for each of the requirements should be clearly defined in vendor agreements. The licensing fees and implementation charges should be paid accordingly. It does not make sense to pay licensing fees for the applications and Database requirements from day one, if the usage comes much later. If necessary activating the processors can be synchronised with the processing requirements.
Agreements and contracts must clearly state that service continuation of the vendors would be subject to satisfactory performance and timely achievement of agreed milestones. This gives Banks the freedom to drop the System Integrator (SI) if they fail to deliver on time. Also, the cost of each project leg should be clearly defined so that if the SI exits midway, the dues do not become controversial. Banks must also ensure diligent documentation of deliveries versus payments made to the SI/vendor.
Let’s also examine a traditional anomaly that provides safe haven to most EDW vendors. As a standard practice almost all Core Banking vendors refrain from providing the Data Dictionary for their systems. This enables them to continually demand their ‘pound of flesh’ from unsuspecting clients. This issue has to be taken up by the Banking fraternity along with the Indian Banks Association (IBA) to ensure that documentation of data, which rightfully belongs to the Banks, is provided to them by the Core Banking vendors and other system vendors at no cost. Interestingly, this is a standard practice the world over. If this practice is not corrected on the home turf, Indian Banks will only continue to bleed.
With requirements defined, what is needed to achieve the desired objective would be a good ETL software, a good Data Mining software, a good OLAP software and a few other tools (depending on the expectations from the EDW). Apart from these, Banks require hardware, that can store up to 10 TB or more of data (for a Bank with about 5000 branches; and this can vary depending on the number of applications in a Bank) and can address the processing requirements for all applications, the people cost for creating the DW Data Models, applications and support.
Banks must also take care to ensure that the cost of Business Continuity / Disaster Recovery planning is kept low. The number of user licenses also must be kept to the minimum. For instance, every Branch or Regional Manager needn’t have a user license and they should only be information Dashboard users. They can access reports relating to the other areas from shared folders. Care should be taken to ensure that only data relevant to the required applications defined by the Bank are extracted and stored.
A third area, where costs can be controlled, is manpower expenses. Many vendors operating in India do not have high quality manpower to support critical activities such as Data Modelling. So activities take longer and Banks keep getting billed. In this scenario, vendors take cover under the ‘unavailability of Data Dictionary’ justification. It would be prudent therefore to go for a fixed cost implementation with a clearly defined and phased delivery schedule. For support, Banks must train their own teams to develop and run ad-hoc reports and Analytics with limited support. There should be a provision in the RFP to reduce vendor support, if necessary or discontinue support, if found unsatisfactory.
A final word of caution: Vendors, especially some of the prominent ‘ivy league’ ones, are adroit at getting agreements signed, which are generally tilted in their favour. Exceptional care must be taken while signing agreements to ensure the Bank’s interest. In fact it may make sense to add the following sentence in vendor agreements – “In the event of a conflict between the RFP, the Purchase Order and the implementation agreement, conditions in the RFP (and responses to the same) will prevail.”
Besides bringing down expenses, these considerations primarily protect the Bank’s interests. If the tools are selected prudently, the cost for a Bank with 4000 to 5000 branches can be brought down by at least 50%.
Try deep learning using MATLAB