DSXchange

I developed a parallel job Reading using External source and it has 7 transformers, 1 join stage and 3 target datasets, 1 Lookup reading using external source stage.
I write the reject records to error log table which is teradata.

The issue is when I ran this job yesterday with 3.9Million records from source. They are joining against 3.5 Millions records from lookup.This Job took 48 mins to run. I want to reduce this time and improve the performance of my Job.

I am using a 4 node default configuration file.

Please suggest me the ways in which i can improve the performance.

Thank you in advance.

If your table is partitioned and local indexed, you may use tablename PARTITION (partitionname). So query will use only one partition.

2nd option: Add some to your where clause, so that it could reduce the number of records.

Thanks.