Parallel job design

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Parallel job design

Post by kpsita »

Hello,

I have a specification to develop a job. in one particular section of the spce I have following condition.

1. I need to select a particular column form a table and if the select is successful then I continue or I stop the process.
2.To continue I need to check if the select produced atleast one row. If I have atleast 1 row then I write these values to a file and if i don't have atleast 1 row then I should goto a different process.
can anyone please tell me how to design this in a parallel job.

Thanks
KPSITA
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Select a column by what?

From source or as a reference ?

If it is from source, you have to look at the link information.

If it is from reference, you are looking for a lookup stage.
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Post by kpsita »

Hi Srinivas,

It is just a select column from a table. And this table is my source.

Thanks,
KPSITA
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

You can call your job from a sequencer.

After the job, check the file whether there are any rows with execute and condition stages.

Depending on the result, branch to your required actions.

I assume you can search and find the stages required.
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Post by kpsita »

Using a sequencer I'll be able to root the job with condition job passed or job failed. But how will I check if there is atleast one row in the file which I extracted.After checking for rows I need to prceed accordingly
Hope I din't confuse...any help
KPSITA
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

1. Lookup stage. Lookup failed rule = "Fail".
2. Allow lookup stage to retrieve multiple rows from reference input. (The rest of your requirement happens automatically.)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Ray,

The o/p expects atleast one row from "source" and not "reference".
battaliou
Participant
Posts: 155
Joined: Mon Feb 24, 2003 7:28 am
Location: London
Contact:

Post by battaliou »

Using DB2 you can issue the following:

Code: Select all

select 1 from dsadm.sample fetch first 1 rows only
This will either fetch 1 or no rows. Usings Ray's lookup technique, you can generate a single row into a lookup and try to get a match and fail it if not.
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.
ShaneMuir
Premium Member
Premium Member
Posts: 508
Joined: Tue Jun 15, 2004 5:00 am
Location: London

Post by ShaneMuir »

Ray's suggestion could be acheived if you used a row generator stage to generate 1 input row containing a dummy key value (say 'X') into a lookup stage. For the reference into the lookup stage you would add a field to your select query which matched your dummy key value. If you then return multiple rows on the link all values which were returned would be output.

Otherwise I would investigate using execute command stages and user variable stages to check your output for values.
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Post by kpsita »

How do I count number of rows in a file using execute stage and then use the condition (rows >0) in a condition stage to proceed my process?

Thanks
KPSITA
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

What have you tried so far ?
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Post by kpsita »

since my post process are sequence, I decided to use execute and condition stage.I am not that familiar with sequencer butafter few research i found that I can call a file in execute stage and then count number of rows in that. Then I'll be able to use a condition stage to call my post process sequencer...any thoughts?
KPSITA
ShaneMuir
Premium Member
Premium Member
Posts: 508
Joined: Tue Jun 15, 2004 5:00 am
Location: London

Post by ShaneMuir »

Its a good start.

You might not necessarily need the condition stage.

You could just use a custom trigger based on your return value from the execute command stage. ie if your command was to check the number of rows and the return value was a number (being the number of rows) you could set your trigger for the outbound link to ReturnValue > 0. This would then only execute when that condition was met.
kpsita
Participant
Posts: 99
Joined: Tue Jul 21, 2009 11:43 pm

Post by kpsita »

since I am not familiar with this stage, I just wanted to know what will be the command used to count rows in execute stage, is it regular rc command?
KPSITA
Post Reply