How can I increase the rows/sec while running the job

[sush26283]

How can I increase the rows/sec while running the job... I have the transaction handling and array size both set to 500 each, but am getting only 10rows/sec..someone please suggest.

Thanks

[jgreve]

I'd suggest starting with telling us how your host-computer doing. If you're already running at 100% CPU utilization, it may be time for a hardware upgrade (I see you're on windows, so I suggest that you start with googling "task manager").

Next, I suggest you say a little about your job design....
How many data sources & targets?
For each of those, how many records, and how big are the records? And where do they come from or go to? (e.g. database, flatfile, webservice...).

What kind of processing are you doing inside your job?
John

[sush26283]

hi...well its simple job with one to one mapping with over 56k records...the source and target are both sql server...with only one source table and one target table

[GJ_Stage]

Hi ,

Can you check and confirm data is pre-sorted or not. Sometime this will make as process slow.

[jgreve]

hi...well its simple job with one to one mapping with over 56k records...the source and target are both sql server...with only one source table and one target table

Over 56K records? So more than 56,000 records but less than what... 57,000 records? At 10/second, that gives you 5,600 seconds; so it takes about 90 minutes to run?

Are your source & target on the same instance of SQL Server? If so, how long does it take to run that query on SQL Server itself? e.g.

insert into test_temp_table select * from source_table;

Replace the red part with the source query you're using in DataStage.
If that is slow, DataStage won't be able to do any better than what SQL Server does.

Can you post your data source SQL statement?

By the way - how was your CPU load doing on the DataStage machine actually running your server job (you did read up on task manager, yes?).

[PaulVL]

Remember that your row per second calculation begins from your job start time, not the time where Row #1 begins to be sent accross to your server.

If you have complex SQL in your connector stage, you might be waiting 5 minutes for your DBMS to handle the SQl and begin returning row #1. Your rows per second factors that 5 mins into your overall math.

Your job startup time is thus a part of that row per second calculation.

[ray.wurlod]

Change the system clock on the server while the job is running.

Rows/sec is an almost totally meaningless metric.