Teradata connector Vs Teradata enterprise stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Chandrathdsx
Participant
Posts: 59
Joined: Sat Jul 05, 2008 11:32 am

Teradata connector Vs Teradata enterprise stage

Post by Chandrathdsx »

I have a job to read from teradata and write into a dataset.
This job is performing much better if I use enterprise stage instead of connector stage.
I am not using bulk option. But, enterprise stage is getting 8 processes where as connector stage using only one process.
Both jobs (connector and enterprise stages) use 8 node config.

Could you please let me know: does enterprise stage is better than connector stage when bulk/utility is not used?
Is there anyway to get more processes/sessions while using the connector stage?
When I use Bulk option in connector, I am getting limit settings enabled to mention max sessions. But, in Immediate access method, it is not behaving similar to enterprise stage.

Log details:
Score from job using teradata connecor (source stage): This job is running @ 800 rows/sec
main_program: This step has 1 dataset:
ds0: {op0[1p] (sequential Teradata_Connector_0)
->eCollectAny
op1[1p] (sequential APT_RealFileExportOperator in Sequential_File_3)}
It has 2 operators:
op0[1p] {(sequential Teradata_Connector_0)
on nodes (
node1[op0,p0]
)}
op1[1p] {(sequential APT_RealFileExportOperator in Sequential_File_3)
on nodes (
node2[op1,p0]
)}
It runs 2 processes on 2 nodes.
##APT_PMsectionLeader: new Section Leader running, PID = 3,547,192
##APT_PMsectionLeader: new Section Leader running, PID = 3,326,120

Score from job using teradata enterprise stage (source stage): This job is running @ 3200 rows/sec
main_program: This step has 1 dataset:
ds0: {op0[8p] (parallel Teradata_Enterprise_0)
[pp] >>eCollectAny
op1[1p] (sequential APT_RealFileExportOperator in Sequential_File_5)}
It has 2 operators:
op0[8p] {(parallel Teradata_Enterprise_0)
on nodes (
node1[op0,p0]
node1[op0,p1]
node1[op0,p2]
node1[op0,p3]
node1[op0,p4]
node1[op0,p5]
node1[op0,p6]
node1[op0,p7]
)}
op1[1p] {(sequential APT_RealFileExportOperator in Sequential_File_5)
on nodes (
node1[op1,p0]
)}
It runs 9 processes on 1 node.

Teradata_Enterprise_0,0: Orchestrate Teradata Read Operator Summary
==========================================
Step Configuration:
-------------------
Total slave sessions: 16
Slave processes: 8
Nodes used: 1

Thanks.
hamzaqk
Participant
Posts: 249
Joined: Tue Apr 17, 2007 5:50 am
Location: islamabad

Post by hamzaqk »

The TPT stage in immediate mode is recommended for small chunks of data as it commits on every few hundered rows which makes it a bit slower than the bulk mode stage. The enterprise stage on the other hand uses bulk fast load with multiple instances which gives a high rate for loading data and hence improved performace as compared to the connector stage.
Teradata Certified Master V2R5
Post Reply