Hash file look up not working

[urs_ravindra]

Hi Dudes,

The following is the flow in my job.

`````````````````Hash File(Having row count in seq file)
``````````````````````|
``````````````````````|
``````````````````````V
Seq File-----------> Transformer------------->Seq File

I want to load all the rows into target file excluding first and last rows. I'm using hash file as look up and getting the row count and using constraint(@INROWNUM>1 and @INROWNUM<>COUNT) to filter first and last rows. But i'm not getting any data into target file.
Using default look up 1=1.
If i remove the contraint all the rows are coming into target, please let me know what is the problem here and how to solve this.

Thanks,
Ravindra

[ArndW]

Ravindra,
I don't know what is wrong with your job, but the design is quite inefficient with the hashed file lookup being performed for each input row {I hope that the hashed file is at least loaded to memory}. You should redesign so that your job is called from a sequence which gets the row count (i.e. through use of a UNIX call to "wc -l {yourfile}") and passes that to the job, then your constraint could read "@INROWNUM#1 AND @INROWNUM#RowCountParameter".

[chulett]

Actually, a single record with a hard-coded key cached in memory is pretty darned efficient for something like this. I've been known to do something like that in other situations.

So Ravindra, to clarify are you saying that:

1) You counted the number of lines in the sequential file.
2) You loaded that number into a single record in the hashed file.
3) There is a key field in the hashed file hard-coded to a '1'.
4) There is a non-key field in the hashed file that holds your row count.
5) The Key Expression for the lookup in the Transformer is also hard-coded to a '1'.
6) Your constraint is "@INROWNUM>1 and @INROWNUM<>LookupLink.COUNT".

If all that is true it should be working. When the job runs, what is the count for the lookup link? It should always match the number of input rows.

[Hadi]

I really can't see any benefit of using hashed file look-up in this job. If it only contains a single constant value, can't you just get the number and hard-code it into the transformer stage? For example, @INROWNUM>1 and @INROWNUM<>1000

[chulett]

The number will change from run to run, so no hard-coding. Yes, if the number is known beforehand it could be passed in as a job parameter and used directly in the constraint. However, this technique should work as well so trying to work thru why it doesn't.

[bmadhav]

Why bother to do a lookup to omit 1st and last row. u can put the foll. constraint in the transformer:

Code: Select all

@INROWNUM > 1 and @INROWNUM <> @OUTROWNUM

If the constraint evaluates to TRUE then u shd all the rows except the 1st & last.

[ray.wurlod]

Can you please explain how that logic excludes the final row?

[chulett]

It doesn't... but then you knew that.

And posting "why bother" messages isn't really all that helpful, either.

[bmadhav]

my bad! u r right, what i said earlier wont work.
Didn't mean to be offensive with the "why bother". sorry!
probably would use a (tail +2) to lop off the last row thru an UNIX command.

[chulett]

No problem. If the OP had asked for other ways to accomplish this, we could go there... or they could simply search, the whole head/tail/sed/awk thing has been posted quite a bit lately. However, since they've posted this technique and asked for help with it, seems to me that would be the best place to start.

[ray.wurlod]

Rats! I thought this old dog could learn a new trick...