Hello,
I have a job that combines data from three datasets using a funnel, this merged data is then loaded into a teradata table using the Teradata Multiload stage. When I tried running the job, it took 8 hours to load 334,000 records, which is obviously way too long a time.
On checking the director logs, i noticed that the time lag between the below two messages was almost the entire runtime, i.e. 8 hours:
11/27/2007 5:50:19 PM Logging delayed metadata.
11/28/2007 1:53:23 AM Requesting delayed metadata
Not sure what these mean and whether they indicate the reason why the job took so long. I checked on the forum and found that various users have encountered these messages in various other stages, including FTP, DB2 etc. but no solution has been put forth.
Can anyone suggest what the problem might be, i.e. what possible reasons might make the job take this long for loading?
The other details of the job are as follows:
Funnel Stage uses "Continuous Funnel" type. From the job-monitor it is seen that all records are funneled in next to no time.
TDMLoadPX Stage:
Load Utility: MultiLoad
Limit Settings: Session Max 2
All other settings (Session Min, Error Limit, Checkpoint etc) set to zero
Any help greatly appreciated!
Thanks,
Kaus
PX job with Teradata Mload taking too long to execute
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 197
- Joined: Sun Jul 15, 2007 11:45 pm
- Location: Prague
I always receive this loggin delayed metadata with MLOAD in PX but jobs run fine, have a look at your report file maybe it contain some logs which will identify the problem, mainly teradata takes time if there are duplicate error or the incoming rows are not properly distrubuted. If all your 300K data goes to one AMP then its a problem but from my point of view it will still be able to insert the data quickly the 8 hours, how many Terdata Nodes are you using and how many AMPS.
-
- Premium Member
- Posts: 197
- Joined: Sun Jul 15, 2007 11:45 pm
- Location: Prague
-
- Participant
- Posts: 24
- Joined: Mon Jul 12, 2004 10:16 am
Re: PX job with Teradata Mload taking too long to execute
a) First Make sure that all your datasets are created with Parallel Option.
b) Make sure that all three datasets has same key and order before passing into funnel stage
c) Instead of Teradata MLoad use Teradata Enterprise stage
b) Make sure that all three datasets has same key and order before passing into funnel stage
c) Instead of Teradata MLoad use Teradata Enterprise stage