Performance Issue

[avunoori]

I developed a parallel job Reading using External source and it has 7 transformers, 1 join stage and 3 target datasets, 1 Lookup reading using external source stage.
I write the reject records to error log table which is teradata.

The issue is when I ran this job yesterday with 3.9Million records from source. They are joining against 3.5 Millions records from lookup.This Job took 48 mins to run. I want to reduce this time and improve the performance of my Job.

I am using a 4 node default configuration file.

Please suggest me the ways in which i can improve the performance.

Thank you in advance.

[chowdhury99]

If your table is partitioned and local indexed, you may use tablename PARTITION (partitionname). So query will use only one partition.

2nd option: Add some to your where clause, so that it could reduce the number of records.

Thanks.