Calling Unix script in parallel from multiple jobs

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
srai
Participant
Posts: 101
Joined: Wed Feb 09, 2005 10:50 pm

Calling Unix script in parallel from multiple jobs

Post by srai »

Hi Team,

We have a UNIX script which read data from database (Teradata) and make the entries in a table.

We are calling same script in more than 100 datastage jobs and those jobs are running in parallel.

Just want to know that if we run the jobs in parallel do we face any issue?

If there is any issue with parallel excution of script Do we need to put any command in UNIX script or in datastage job while calling the script so that it can be executed in parallel.

Regards,
Santosh
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

UNIX scripts can be run in parallel. What we cannot guess is exactly what SQL you are using in your script and if that might have problems running in parallel. Can you specify "for read only" in Teradata SELECTs so that no update locks are taken?
srai
Participant
Posts: 101
Joined: Wed Feb 09, 2005 10:50 pm

Post by srai »

ArndW wrote:UNIX scripts can be run in parallel. What we cannot guess is exactly what SQL you are using in your script and if that might have problems running in parallel. Can you specify "for read only" in Teradata SELECTs so that no update locks are taken?
Actually Unix is reading from a teradata table and after execution it is updating the Audit table based on the Job Audit id column.
But when we execute multiple jobs in paralle there will be multiple update statement getiing executed at DB level.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

In that case you need to ensure that you don't get deadlocks when running your script in parallel by either committing everything immediately and/or by using different records in each script.
dsa
Participant
Posts: 37
Joined: Sun Oct 10, 2010 7:52 am

Post by dsa »

Hi,

You have marked this topic as resolved. Could you please let us know how did you make sure that deadlock situation doesn't come?
Post Reply