Warehouse design rant

An anonymous poster to Tom Kyte’s Blog shared a tale of woe about data warehouse performance. Well, I felt for that poster – how can a developer put their hand on their heart (or wallet, if they are an external consultant) and say in effect “design does not matter, the database will be able to handle it”

Of course design matters; and getting it right is important for the success of any development project. For data warehousing this means an holistic approach – the user base wants to exploit their data capital for business advantage, so they need a system that:

1) Provides the correct results 2) Provides those results in a reasonable timescale 3) Allows new data to be added to the DW, again in an acceptable batch window

Get any of these wrong and your project will be seen as a failure in the eyes of some or all of your users. Remember, it is these users that ‘pay’ for the systems, the days of IT departments sponsoring a development project to keep staff ‘gainfully employed’ are long gone. Data warehouses should be seen as a strategic asset and therefore be designed for sustainability and longevity – this is not a technology area for the ‘quick fix’ or those who shun best practice.

Oh, and one more part of my rant – vendor neutral database design does not work in a data warehouses – Oracle has a fantastic set of performance enhancing features for large databases, omit using them appropriately and you will struggle to deliver.