Perfromance issue while loading into SQL Server 2008

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Narayan Reddy
Premium Member
Premium Member
Posts: 5
Joined: Thu Oct 20, 2011 1:35 am
Location: Bangalore

Perfromance issue while loading into SQL Server 2008

Post by Narayan Reddy »

Hi,

We have ~12 million(1.8 GB) of records which need to be loaded into MS SQL Server 2008.

While loading initially(say for 2 million records) it will load with 10000 rows/sec but gradually it will reduce to few hundreds. We think it can be Network or buffer related issue.Network speed is 100 Mbps.

Please guide us in increasing the performance.

Thanks,
Reddy
Reddy
SURA
Premium Member
Premium Member
Posts: 1229
Joined: Sat Jul 14, 2007 5:16 am
Location: Sydney

Re: Perfromance issue while loading into SQL Server 2008

Post by SURA »

Find the way to use BCP utility.

DS User
Narayan Reddy
Premium Member
Premium Member
Posts: 5
Joined: Thu Oct 20, 2011 1:35 am
Location: Bangalore

Re: Perfromance issue while loading into SQL Server 2008

Post by Narayan Reddy »

Is any other way around like setting some parameters or Array size in ODBC Connector to increase the performance?
Reddy
Rakesh311
Participant
Posts: 36
Joined: Wed Aug 04, 2010 10:53 pm
Location: Banglore
Contact:

Re: Perfromance issue while loading into SQL Server 2008

Post by Rakesh311 »

Hi guys,

Is there any settings(tuning) to be done for increasing number of rows inserted per second while directly loading to the SQL SERVER database.

Thanks
Rakesh
rAKESH
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The "normal" ones are there - Array Size, Autocommit Mode / Record count, etc. What settings are you using? I'd also be curious what "Write Mode" you are using.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply