I developed a parallel job Reading using External source and it has 7 transformers, 1 join stage and 3 target datasets, 1 Lookup reading using external source stage.
I write the reject records to error log table which is teradata.
The issue is when I ran this job yesterday with 3.9Million records from source. They are joining against 3.5 Millions records from lookup.This Job took 48 mins to run. I want to reduce this time and improve the performance of my Job.
I am using a 4 node default configuration file.
Please suggest me the ways in which i can improve the performance.
Thank you in advance.
Performance Issue
Moderators: chulett, rschirm, roy
Performance Issue
Thanks,
Avunoori.
Avunoori.
-
- Participant
- Posts: 43
- Joined: Thu May 29, 2008 8:41 pm
Re: Performance Issue
If your table is partitioned and local indexed, you may use tablename PARTITION (partitionname). So query will use only one partition.
2nd option: Add some to your where clause, so that it could reduce the number of records.
Thanks.
2nd option: Add some to your where clause, so that it could reduce the number of records.
Thanks.