TD API Load Performance
Posted: Mon Nov 09, 2009 10:01 am
Hello Everyone,
I am reading from a text file into a transformer, where I am defaulting a couple of columns to null (SetNull()) and one other column to CurrentTimestamp() which then flows to Terada API stage to do Upserts. On any given day, the number of records in the text file are not more than 500 records.
On average, it takes about 3 to 5 hours in the step "Logging Delayed Metadata" and eventually gets to "Requesting Delayed Metadata"
I am looking at the run time stats in Teradata and there is no huge CPU are I/O consumtion for this job?
Having hard time to understand on where is it spinning its wheels for about 3 to 5 hours.
Please shed some light and let me know if you need more information on this.
Thanks
I am reading from a text file into a transformer, where I am defaulting a couple of columns to null (SetNull()) and one other column to CurrentTimestamp() which then flows to Terada API stage to do Upserts. On any given day, the number of records in the text file are not more than 500 records.
On average, it takes about 3 to 5 hours in the step "Logging Delayed Metadata" and eventually gets to "Requesting Delayed Metadata"
I am looking at the run time stats in Teradata and there is no huge CPU are I/O consumtion for this job?
Having hard time to understand on where is it spinning its wheels for about 3 to 5 hours.
Please shed some light and let me know if you need more information on this.
Thanks