Problem performance datawarehouse with lots of indexes

Posted by Lieven Cardoen on Stack Overflow See other posts from Stack Overflow or by Lieven Cardoen
Published on 2009-12-10T09:02:58Z Indexed on 2010/05/18 2:40 UTC
Read the original article Hit count: 363

Filed under:

database

|

indexes

|

data-warehouse

Our product takes tests of some 350 candidates at the same time. At the end of the test, results for each candidate are moved to a datawarehouse full of indexes on it. For each test there's some 400 records to be entered in datawarehouse. So 400 x 350 is a lot of records. If there are not much records in the datawarehouse, all goes well. But if there are already lots of records in the datawarehouse, then a lot of inserts fail...

Is there a way to have indexes that are only rebuild at the end of the day or isn't that the real problem? Or how would you solve this?

© Stack Overflow or respective owner

Related posts about database

SQL SERVER Retrieve and Explore Database Backup without Restoring Database Idera virtual database

as seen on Dot net Slackers - Search for 'Dot net Slackers'
I recently downloaded Ideras SQL virtual database, and tested it. There are a few things about this tool which caught my attention.My ScenarioIt is quite common in real life that sometimes observing or retrieving older data is necessary; however, it had changed as time passed by. The full database… >>> More
Cloning A Database On The Same Server Using Rman Duplicate From Active Database

as seen on Oracle Blogs - Search for 'Oracle Blogs'
To clone a database using Rman we used to require an existing Rman backup, on 11g we can clone databases using the "from active" database option. In this case we do not require an existing backup, the active datafiles will be used as the source for the clone. In order to clone with the source database… >>> More
cPickle ImportError: No module named multiarray

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I'm using cPickle to save my Database into file. The code looks like that: def Save_DataBase(): import cPickle from scipy import * from numpy import * a=Results.VersionName #filename='D:/results/'+a[a.find('/')+1:-a.find('/')-2]+Results.AssType[:3]+str(random.randint(0,100))+Results.Distribution+"… >>> More
SQL SERVER – 2008 – Introduction to Snapshot Database – Restore From Snapshot

as seen on SQL Authority - Search for 'SQL Authority'
Snapshot database is one of the most interesting concepts that I have used at some places recently. Here is a quick definition of the subject from Book On Line: A Database Snapshot is a read-only, static view of a database (the source database). Multiple snapshots can exist on a source database and… >>> More
OTN ???? ?????? ???????

as seen on Oracle Blogs - Search for 'Oracle Blogs'
Database ?? Database ??????? Database ?????????? Java WebLogic Server/????????·???? SOA/BPM/????? ???????/???? ID??/?????? ?????EPM/BI EPM/BI ??????? EPM/BI ???? OS/??? ???? ????? MySQL Database ?? ???? ?? ????????? ??? ?? ORACLE MASTER… >>> More

Related posts about indexes

Stairway to SQL Server Indexes: Step 1, Introduction to Indexes

as seen on SQL Server Central - Search for 'SQL Server Central'
Indexes are the database objects that enable SQL Server to satisfy each data access request from a client application with the minimum amount of effort, resulting in the maximum performance of individual requests while also reducing the impact of one request upon another. Prerequisites: Familiarity… >>> More
When SQL Server Nonclustered Indexes Are Faster Than Clustered Indexes

as seen on SQL Server Central - Search for 'SQL Server Central'
SQL Server Clustered indexes can have enormous implications for performance of operations on a table. But are there times when a SQL Server non-clustered index would perform better than a clustered index for the same operation? Are there any trade-offs to consider? Check out this tip to learn more… >>> More
text indexes vs integer indexes in mysql

as seen on Stack Overflow - Search for 'Stack Overflow'
Hey, I have always tried to have an integer primary key on a table no matter what. But now I am questioning if this is always necessary. Let's say I have a product table and each product has a globally unique SKU number - that would be a string of say 8-16 characters. Why not make this the PK… >>> More
SQL SERVER – Disabled Index and Update Statistics

as seen on SQL Authority - Search for 'SQL Authority'
When we try to update the statistics, it throws an error as if the clustered index is disabled. Now let us enable the clustered index only and attempt to update the statistics of the table right after that. Have you ever come across the situation where a conversation never gets over and it continues… >>> More
Macro doesn't work in the function.

as seen on Stack Overflow - Search for 'Stack Overflow'
I have problems with following code: http://lisper.ru/apps/format/96 The problem is in "normalize" function, which does not work. It fails on the fifth line: (zero-p a indexes i) (defun normalize (a &optional indexes i) "Returns normalized A." (pragma (format t "Data=~A ~A ~A" a indexes… >>> More