Database connections in datastage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
skathaitrooney
Participant
Posts: 103
Joined: Tue Jan 06, 2015 4:30 am

Database connections in datastage

Post by skathaitrooney »

Hello Experts,

Just had a query.

Suppose my jobs runs on a head node and a compute node.

Now if i am using a database in the job, will the network sockets for the database be opened from the head node or the compute node?

Currently i am seeing sockets opened for both head node and compute node.

i.e. from where exactly are the database connections made?
electajay
Participant
Posts: 36
Joined: Thu Apr 15, 2010 11:19 am

Re: Database connections in datastage

Post by electajay »

If i understand correctly. you are trying to connect to database ex: oracle from Datastage server A (head Node) server B (Compute node). yes you need to open the firewall and ports from both the Servers A and B and update the tnsnames.ora file on both the Servers A and B.

I think Database connections are made from Conductor node (Head Node)
lets see what others say.
A Kumar
AshishDevassy
Participant
Posts: 32
Joined: Fri Nov 07, 2014 11:18 am

Post by AshishDevassy »

from what i remember , connections are made thru the head nodes.
that's wierd ?
skathaitrooney
Participant
Posts: 103
Joined: Tue Jan 06, 2015 4:30 am

Post by skathaitrooney »

Ashsish/electajay,

Let me explain further, suppose i am using a DB2 DB in my job. I am connecting to db server using port say 65432. Now when i do a

Code: Select all

netstat -a|grep 65432
on both head nodes and compute node, i can see connections established.

So i was wondering where exactly are the connections made from? Head node or compute node.
Also the section leaders for the connector stage runs on compute nodes, so shouldnt the DB connections be made from the compute nodes?

Currently i can see DB connections on both head and compute node.
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

"Validate Column type and size mismatches" happen off the head node I believe.

Some connector stages have that as options.
AshishDevassy
Participant
Posts: 32
Joined: Fri Nov 07, 2014 11:18 am

Post by AshishDevassy »

here is something you can try.
I am sure that the DB2 Database must be cataloged on both the head and the compute node. UnCatalog the database in question from all the compute nodes and run the job.
And when you run send the job to these compute nodes.
that's wierd ?
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

umm that's a little harsh and unwise since that could mess up your developers.

I vote nay on that last recommendation.

You could do it in an Admin Only environment, but not a live DEV/TEST/PROD where users will chew you out.
skathaitrooney
Participant
Posts: 103
Joined: Tue Jan 06, 2015 4:30 am

Post by skathaitrooney »

Hey guys,

I just asked my DB2 DBA for a application snap, in that i can see the connections being made from the compute nodes only(that are in Executing state).

Also, on "VIEW Data", connections are made from the head node

:)
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

"View Data" at design time uses a different connection method than connections made at run time, so that would make sense.

The DB2 Connector also has a setting (not sure its value, as I'm not on a grid): In the Alternate conductor settings field, specify whether you want to provide alternate connection settings to use on the conductor node. The default value is No.

Also, here is some interesting reading on using direct connections:
http://www-01.ibm.com/support/knowledge ... tions.html
Choose a job you love, and you will never have to work a day in your life. - Confucius
Post Reply