job performance
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 397
- Joined: Wed Apr 12, 2006 2:28 pm
- Location: Tennesse
job performance
Hi all , i am running a parallel job as
[seq----transformer----sqlserver]
i have divided the input file data as 3 million / run (total rows 12 million in original file) though each file has same meta data , every time i run the performance get reduce like
1st runs statistics is 5000 rows/sec
2nd run--------------1000 rows/ sec
3rd run --------------500 rows/sec
4th run --------------250 rows/sec
i dnt knw y the performance is getting reduce for the successive runs can any one plz let me knw why this happens and how can we increase the performance.
Thanks in advance
[/code]
[seq----transformer----sqlserver]
i have divided the input file data as 3 million / run (total rows 12 million in original file) though each file has same meta data , every time i run the performance get reduce like
1st runs statistics is 5000 rows/sec
2nd run--------------1000 rows/ sec
3rd run --------------500 rows/sec
4th run --------------250 rows/sec
i dnt knw y the performance is getting reduce for the successive runs can any one plz let me knw why this happens and how can we increase the performance.
Thanks in advance
[/code]
Last edited by samsuf2002 on Fri Oct 13, 2006 11:10 am, edited 1 time in total.
hi sam here
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 397
- Joined: Wed Apr 12, 2006 2:28 pm
- Location: Tennesse
-
- Premium Member
- Posts: 397
- Joined: Wed Apr 12, 2006 2:28 pm
- Location: Tennesse
They are talking about SQL Server partitioning where you break one table up into to many tables but they still act as one. You usually want to break a table by either a reporting field or a date field or both. Some people want to partition by state or country for reporting purposes. Another way would be to have a partition for each month based on sales date or some date field. ETL can usually load into one partition therefore the load times increase because the one partition table is smaller than one huge table. It also helps in archiving because you can just remove one partition.
Mamu Kim