can any one tell example for datastage functions

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kstechno
Participant
Posts: 6
Joined: Tue Apr 05, 2011 9:14 pm

can any one tell example for datastage functions

Post by kstechno »

For example we have Customer table

In customer table
we have a c_id and c_name

c_id c_ name
11 a
11 b
22 c
33 d
44 e
22 f
11 g
33 h
44 i


And my question is
There should be 2 targets and
in Target 1 I need all uniques ids
and in Target 2 I need all duplicates

Can any one send me the full condition we need to write.

I have taken seqfile as a source to read the data
and I have taken sort stage to sort the data
and from there i have taken transformer stage
and I need to write condition there.

Tell me the condition I need to write exactly.

To get the answet in the Target .

Thanks in advance
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Add a 'key change column' to the Sort stage and then examine the values that come from it. You should easily be able to tell what constraint expression to use for each target.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kstechno
Participant
Posts: 6
Joined: Tue Apr 05, 2011 9:14 pm

Post by kstechno »

Yeah this is fine.
Can you tell me the condition to write in transformer stage.
I m not getting correct output when writing condition ( EX if then else condition )
Plz give me the condition with if the else condition .

Thank you
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

Have you read the documentation for the Sort stage to understand what the keyChange column option provides you and the values it will contain? It's in the Parallel Job Developer Guide.

As a quick exercise, imagine you have a column on your records which uses two distinct values to indicate whether or not a record is the first record of a group of duplicates. In a constraint within a transformer, how would YOU write the logic to select only the first record? In another constraint, how would YOU write the logic to select only the duplicates?

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

There's no "if-then-else" needed here. You have two output links and always fully populate both sets of columns with the same data. The key is the constraint so that only one or the other 'fires' for each record and the constraint can make use of the Key Change column.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kstechno
Participant
Posts: 6
Joined: Tue Apr 05, 2011 9:14 pm

Post by kstechno »

Thank you very much chulett for your reply.

I got it.

And I need one more answer.

I need the data in the output as below

c_id , c_name

11 a,b,g
22 c,f
33 d,h
44 e,i

Plz reply what condition i can write to get this output.

Thanks in advance
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Vertical pivot. Search DSXchange for examples.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
vishal_rastogi
Participant
Posts: 47
Joined: Thu Dec 09, 2010 4:37 am

Post by vishal_rastogi »

use the aggregator stage
group it by according to the key column
count the no. of rows by using the count row
now in transformes stage use the constraint .....linkcount = 1 (for uniques)
and linkcount > 1 (for duplicate)
Vish
Post Reply