Lookup Stage Issue

sridharvis · Post by **sridharvis** » Tue Oct 13, 2009 5:07 am

Hi,

I have a odbc enterprise stage as a stream link to a look up stage and sequential File stage as a reference link to a look up stage.

In the lookup stage there are no constraints or conditions, i had set the Lookup Failure option to Drop

I had left the partitioning option to auto on both the stream and reference, assuming the default partioning for lookup is entire.

My ODBC source had 28,000 records and the reference seq File had 175 records, when i try to run it in development environment it fetches me 175 records, but when the same job is executed in testing environment it processes only 170 records and rejects 5.

To rectify it , In testing i explicitly change the input partioning to Entire on the reference link then i found all 175 records successfully passing through , My question why were the 5 records rejected in testing , but not in development , in addition to it why were the records processed once i set the partioning to entire.Please help me

roy · Post by **roy** » Tue Oct 13, 2009 6:37 am

Hi,
As far as I recall if you were preserving the partitioning you would have retrieved all 175, assuming all jobs use the same partitioning algorithm and the same config file.
Is there any chance you run the jobs using different cnfiguration files?
(though misssing 5 of 175 records would be odd even if this is the case)

When you specify "entire" you say that DS should search for records in the entire lookup file inspite of partitioning.
Seems that the partitioning done was not the same in the end to end process you run.
I guess answering this exactly would require examining your jobs.

IHTH (I Hope This Helps),

sridharvis · Post by **sridharvis** » Tue Oct 13, 2009 8:26 am

Roy,

The job is pretty much simple. I have a ODBC Enterprise stage, where i have a char(5) column and a sequential File stage that also has a char(5) column. I havent changed the preserve partioning tab at all in my job, Both the odbc (stream) and seqFile(reference) goes to a look up , In the lookup both are set to Auto Partition and from the lookup i load the target table

The config file in dev is 2 node and in testing is 2 node.

Nothing much is happening in job other than this.

I have a question i find the field with which i perform lookup is char(5), but some in the source(seq,odbc) are char(4). 'ABCD ' and 'ABCD '. Will that cause a problem during lookup ?

roy · Post by **roy** » Tue Oct 13, 2009 11:12 am

If you had a matching problem you wouldn't have got 175 successfull lookup what ever you did.
I would try to perform lookup on fields with the same metadata.

Just making shure I understand you, your main stream is a seq file and your performing the lookup against an ODBC stage

If this odd behaviour persists try rebuilding your job from scratch and keep us posted.

IHTH,

sridharvis · Post by **sridharvis** » Wed Oct 21, 2009 3:52 am

Nope. My stream link is odbc and reference(lookup) link is Seq File.

Sure I will let you know by today. I would build my job from scratch and investigate the issue.