Compare two Output Columns

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vinodn
Charter Member
Charter Member
Posts: 93
Joined: Tue Dec 13, 2005 11:00 am

Compare two Output Columns

Post by vinodn »

Hi all,

I am having a Tailer record in which I will have a count of the total file and I need to compare this count with Datastage count from Transformer Stage. So which stage should I use to compare both the outputs.

Actually I have taken Tail Stage for Trailer Count and for Datastage Count I have done it through Aggregator. Now I want to compare this two fields. If this two fields are INCORRECT then I need to Abort the job.

I tried using Compare Stage but it is throwing following error
Compare_85: The number of attached output data sets (0) differs from APT_Operator::setOutputDataSets() arg (1).

Thanxs in Advance
bkumar103
Participant
Posts: 214
Joined: Wed Jul 25, 2007 2:29 am
Location: Chennai

Post by bkumar103 »

If your requirement is to just verify the records in tha tailer with the records then this can be achieved simply in the before job subroutine.
This can be implemented in shell script as follow.
Suppose file1 is the file with following format
===file1====
record1
record2
record3
3

now you have to verify that the value in the tailer record with the no of records in the file.

var1=`tail -1 file1` -- cut the tailer record represent the count
var2='wc -l file1`
var2=`expr $var2 - 1` -- count the no of lines, reduces by 1 because last line is the tailer record.

now both variable can be compared for equality.
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Re: Compare two Output Columns

Post by JoshGeorge »

You don't need an extra stage for this, you can do this in transformer itself.

AGG_Output.Count = Datastage.Count :? (Datastage.Count what will be that? )
vinodn wrote: I am having a Tailer record in which I will have a count of the total file and I need to compare this count with Datastage count from Transformer Stage. So which stage should I use to compare both the outputs.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
Post Reply