Difference between Teradata Enterprise & Teradata API st

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Shamitha_Shetty
Participant
Posts: 4
Joined: Sun Jul 15, 2007 11:59 pm

Difference between Teradata Enterprise & Teradata API st

Post by Shamitha_Shetty »

Which Teradata Stage is more efficient when there is huge input data for extraction?Input volume:approximately 450 million recs.
which Teradata stage (Enterprise or API) helps extraction of data more efficiently?
Also after extraction from any of the above is done,i need to check if the extraction returns no rows then interface should abort.how can this be done?
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Definitely Teradata (TD) API is not the choice if you have millions of records. TD EE (Enterprise) when used as source calls FASTEXPORT (TD Unload Utility). For explicit exports or big data pulls TD EE can work fine, or you can use the MLOAD stage (which does Fastexport if it is a source stage).
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

To check whether no rows were extracted before proceeding you need a modular design - one module (job) to perform the extraction into a staging area (Data Set), and a later module to process the data from that staging area. Perform your check before starting the second module.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Shamitha_Shetty
Participant
Posts: 4
Joined: Sun Jul 15, 2007 11:59 pm

Post by Shamitha_Shetty »

Yeah,for the volume of input data to extract,modular design is suitable.but how do we go about checking if the extraction is empty.Taking a count of the table is not appropriate looking at the huge volume of data.Is there any other alternative?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Try selecting just the first row from the table.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
hamzaqk
Participant
Posts: 249
Joined: Tue Apr 17, 2007 5:50 am
Location: islamabad

Post by hamzaqk »

TD API reads the records sequentially and is preferred when you have to read small number of records although it does not have any limits pertaining to TD utilities.


TD Ent stage is the only stage which can read and write in parallel and is preferred for large volume of data. it uses TeraRead and Terawrite or in other words fast export and load protocols and thus has the same limitations which the TD utilities have.

Teradata Certfied Master V2R5
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Interesting... I think TD API have the option to run in parallel mode also. One can write/insert records using TD API in parallel mode and it gives much better performance than using the same in sequential mode.
hamzaqk wrote:TD API reads the records sequentially and is preferred when you have to read small number of records although it does not have any limits pertaining to TD utilities.
TD EE is not the 'only' parallel stage I think. TD Mload is also widely used.
hamzaqk wrote: TD Ent stage is the only stage which can read and write in parallel and is preferred for large volume of data. it uses TeraRead and Terawrite or in other words fast export and load protocols and thus has the same limitations which the TD utilities have.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
Post Reply