PX job with Teradata Mload taking too long to execute
Posted: Wed Nov 28, 2007 10:30 am
Hello,
I have a job that combines data from three datasets using a funnel, this merged data is then loaded into a teradata table using the Teradata Multiload stage. When I tried running the job, it took 8 hours to load 334,000 records, which is obviously way too long a time.
On checking the director logs, i noticed that the time lag between the below two messages was almost the entire runtime, i.e. 8 hours:
11/27/2007 5:50:19 PM Logging delayed metadata.
11/28/2007 1:53:23 AM Requesting delayed metadata
Not sure what these mean and whether they indicate the reason why the job took so long. I checked on the forum and found that various users have encountered these messages in various other stages, including FTP, DB2 etc. but no solution has been put forth.
Can anyone suggest what the problem might be, i.e. what possible reasons might make the job take this long for loading?
The other details of the job are as follows:
Funnel Stage uses "Continuous Funnel" type. From the job-monitor it is seen that all records are funneled in next to no time.
TDMLoadPX Stage:
Load Utility: MultiLoad
Limit Settings: Session Max 2
All other settings (Session Min, Error Limit, Checkpoint etc) set to zero
Any help greatly appreciated!
Thanks,
Kaus
I have a job that combines data from three datasets using a funnel, this merged data is then loaded into a teradata table using the Teradata Multiload stage. When I tried running the job, it took 8 hours to load 334,000 records, which is obviously way too long a time.
On checking the director logs, i noticed that the time lag between the below two messages was almost the entire runtime, i.e. 8 hours:
11/27/2007 5:50:19 PM Logging delayed metadata.
11/28/2007 1:53:23 AM Requesting delayed metadata
Not sure what these mean and whether they indicate the reason why the job took so long. I checked on the forum and found that various users have encountered these messages in various other stages, including FTP, DB2 etc. but no solution has been put forth.
Can anyone suggest what the problem might be, i.e. what possible reasons might make the job take this long for loading?
The other details of the job are as follows:
Funnel Stage uses "Continuous Funnel" type. From the job-monitor it is seen that all records are funneled in next to no time.
TDMLoadPX Stage:
Load Utility: MultiLoad
Limit Settings: Session Max 2
All other settings (Session Min, Error Limit, Checkpoint etc) set to zero
Any help greatly appreciated!
Thanks,
Kaus