Which Teradata Stage is more efficient when there is huge input data for extraction?Input volume:approximately 450 million recs.
which Teradata stage (Enterprise or API) helps extraction of data more efficiently?
Also after extraction from any of the above is done,i need to check if the extraction returns no rows then interface should abort.how can this be done?
Difference between Teradata Enterprise & Teradata API st
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 4
- Joined: Sun Jul 15, 2007 11:59 pm
-
- Participant
- Posts: 612
- Joined: Thu May 03, 2007 4:59 am
- Location: Melbourne
Definitely Teradata (TD) API is not the choice if you have millions of records. TD EE (Enterprise) when used as source calls FASTEXPORT (TD Unload Utility). For explicit exports or big data pulls TD EE can work fine, or you can use the MLOAD stage (which does Fastexport if it is a source stage).
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
To check whether no rows were extracted before proceeding you need a modular design - one module (job) to perform the extraction into a staging area (Data Set), and a later module to process the data from that staging area. Perform your check before starting the second module.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 4
- Joined: Sun Jul 15, 2007 11:59 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
TD API reads the records sequentially and is preferred when you have to read small number of records although it does not have any limits pertaining to TD utilities.
TD Ent stage is the only stage which can read and write in parallel and is preferred for large volume of data. it uses TeraRead and Terawrite or in other words fast export and load protocols and thus has the same limitations which the TD utilities have.
Teradata Certfied Master V2R5
TD Ent stage is the only stage which can read and write in parallel and is preferred for large volume of data. it uses TeraRead and Terawrite or in other words fast export and load protocols and thus has the same limitations which the TD utilities have.
Teradata Certfied Master V2R5
-
- Participant
- Posts: 612
- Joined: Thu May 03, 2007 4:59 am
- Location: Melbourne
Interesting... I think TD API have the option to run in parallel mode also. One can write/insert records using TD API in parallel mode and it gives much better performance than using the same in sequential mode.
TD EE is not the 'only' parallel stage I think. TD Mload is also widely used.hamzaqk wrote:TD API reads the records sequentially and is preferred when you have to read small number of records although it does not have any limits pertaining to TD utilities.
hamzaqk wrote: TD Ent stage is the only stage which can read and write in parallel and is preferred for large volume of data. it uses TeraRead and Terawrite or in other words fast export and load protocols and thus has the same limitations which the TD utilities have.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>