Take advantage of multiple cores executing SQL statements

Posted by willvv on Stack Overflow See other posts from Stack Overflow or by willvv
Published on 2009-03-31T18:51:08Z Indexed on 2012/08/30 21:38 UTC
Read the original article Hit count: 131

Filed under:

sql

|

insert

|

parallel-processing

I have a small application that reads XML files and inserts the information on a SQL DB.

There are ~ 300 000 files to import, each one with ~ 1000 records.

I started the application on 20% of the files and it has been running for 18 hours now, I hope I can improve this time for the rest of the files.

I'm not using a multi-thread approach, but since the computer I'm running the process on has 4 cores I was thinking on doing it to get some improvement on the performance (although I guess the main problem is the I/O and not only the processing).

I was thinking on using the BeginExecutingNonQuery() method on the SqlCommand object I create for each insertion, but I don't know if I should limit the max amount of simultaneous threads (nor I know how to do it).

What's your advice to get the best CPU utilization?

Thanks

© Stack Overflow or respective owner

Related posts about sql

SQL SERVER – Concat Strings in SQL Server using T-SQL – SQL in Sixty Seconds #035 – Video

as seen on SQL Authority - Search for 'SQL Authority'
Concatenating string is one of the most common tasks in SQL Server and every developer has to come across it. We have to concat the string when we have to see the display full name of the person by first name and last name. In this video we will see various methods to concatenate the strings. SQL… >>> More
SQL SERVER – Concat Function in SQL Server – SQL Concatenation

as seen on SQL Authority - Search for 'SQL Authority'
Earlier this week, I was delivering Advanced BI training on the subject of “SQL Server 2008 R2″. I had great time delivering the session. During the session, we talked about SQL Server 2010 Denali. Suddenly one of the attendees suggested his displeasure for the product. He said, even though… >>> More
Error with SQL Server Setup 2012 on Windows 2012

as seen on Server Fault - Search for 'Server Fault'
I am trying to install SQL Server on Windows 2012. I was able to finally get the wizard up and running after making some changes on the server, but now it fails no matter what I do with the following error: TITLE: SQL Server Setup failure. SQL Server Setup has encountered the following error: … >>> More
How can I detect which version of SQL (eg SQL 2008 or SQL Azure)

as seen on Stack Overflow - Search for 'Stack Overflow'
I need to detect which version of SQL I am dealing with to perorm various tasks, I need specifically detect if I am on SQL 2008 or SQL Azure. How can I do this with detection code written in SQL? >>> More
Nested SQL Select statement fails on SQL Server 2000, ok on SQL Server 2005

as seen on Stack Overflow - Search for 'Stack Overflow'
Here is the query: INSERT INTO @TempTable SELECT UserID, Name, Address1 = (SELECT TOP 1 [Address] FROM (SELECT TOP 1 [Address] FROM [UserAddress] ua INNER JOIN UserAddressOrder uo ON ua.UserID = uo.UserID WHERE ua.UserID = u.UserID ORDER BY uo.AddressOrder ASC) q ORDER BY AddressOrder… >>> More

Related posts about insert

Oracle pl\sql question for my homework in oracle 11G class [migrated]

as seen on Programmers - Search for 'Programmers'
I am new to oracle 11G programming and i have run into a tough situation with pl\sql funtions and automation. I ame unsure how to create the function for the automation of Registration system for a College registration system. Here is what i want to do. I want to automate the registrations system… >>> More
SQL Server giving a weird conversion error

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello Everyone I'm stuck and any help you can give me is greatly appreciated. create table stackoverflow_rules(myguid uniqueidentifier primary key, myvarchar50 varchar(50)) insert into stackoverflow_rules(myguid, myvarchar50) values('27C4CF31-2C4C-4C78-BBDC-2D0FDAA891CF','9985276') insert into… >>> More
Run MySQL INSERT Query multiple times (insert values into multiple tables)

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, basically, I have 3 tables; users and projects (which is a many-to-many relationship), then I have 'usersprojects' to allow the one-to-many formation. When a user adds a project, I need the project information stored and then the 'userid' and 'projectid' stored in the usersprojects table. It sounds… >>> More
SQL SERVER – Performance Comparison – INSERT TOP (N) INTO Table – Using Top with INSERT

as seen on SQL Authority - Search for 'SQL Authority'
Recently I wrote about SQL SERVER – INSERT TOP (N) INTO Table – Using Top with INSERT I mentioned about how TOP works with INSERT. I have mentioned that I will write about the performance in next article. Here is the performance comparison of the two options. It is clear that performance of the Insert… >>> More
SQL SERVER – INSERT TOP (N) INTO Table – Using Top with INSERT

as seen on SQL Authority - Search for 'SQL Authority'
During my recent training at one of the clients, I was asked regarding the enhancement in TOP clause. When I demonstrated my script regarding how TOP works along with INSERT, one of the attendees suggested that I should also write about this script on my blog. Let me share this with all of you and… >>> More