DSXchange

WHY won't you do what I asked and post the result?

ray.wurlod wrote:

I wrote:Add a reject link to the Lookup stage so that you can see exactly what values were sought.

What happened when you tried that? What value was in the date column that you used for a key, and subsequently was sent to the reject link?

I have incorporated the date field also on which the look up was done in the output file , but that is not getting populated , that is the date field and the id fields are blank in the output field. So it proves that look up is basically failing.

Until you create the reject link and report some values for the date column from it, you will be getting no more help from me on this thread.

As Ray said collect those failed(rejected) records in rejected link. To do this set Lookup Failure(in Constraints tab) to Reject and drag reject link from LookUp Stage.

And see why are those rejected ?

ray.wurlod wrote:Until you create the reject link and report some values for the date column from it, you will be getting no more help from me on this thread.

I have created the reject link and all the data from the primary link is going into the reject link.

.. and do the dates (that you use as keys in the Lookup stage) look like the expected dates?

Now you need to check the dates in the reference Data Set, and verify the values of these.

Diagnosis is a painstaking, step by step process. You must proceed systematically, eliminating one possibility at a time.

ray.wurlod wrote:.. and do the dates (that you use as keys in the Lookup stage) look like the expected dates?

Now you need to check the dates in the reference Data Set, and verify the values of these.

Diagnosis is a painstaking, step by step process. You must proceed systematically, eliminating one possibility at a time.

Not all the dates are matching but there are some natching dates and for those the data should be there in the output file eliminating the non matching data.

Should be, would be, could be...

Show us (a) what the dates look like and (b) the data types (not just VarChar, the precision also). Show us the record schema from the stream input link and the reference input link to the Lookup stage. Tell us which columns are identified as Key. Otherwise we're guessing, and wasting everyone's time.

ray.wurlod wrote:Should be, would be, could be...

Show us (a) what the dates look like and (b) the data types (not just VarChar, the precision also). Show us the record schema from the stream input link and the reference input link to the Lookup stage. Tell us which columns are identified as Key. Otherwise we're guessing, and wasting everyone's time.

may be the reference file was corrupted , i have changed the reference file , now the job is running file but the output file is getting populated with even those dates that do not have the matching data with the id field having empty values.

Doubtless your "Lookup Failed" rule is "Continue" and the Lookup stage is therefore behaving as documented.

ray.wurlod wrote:Doubtless your "Lookup Failed" rule is "Continue" and the Lookup stage is therefore behaving as documented.

Yes the "Lookup Failed" rule is "Continue" and when i set it to "Fail" the job is getting aborted.

ray.wurlod wrote:Doubtless your "Lookup Failed" rule is "Continue" and the Lookup stage is therefore behaving as documented.

"Lookup Failed" rule is set to "Drop" and then the job is running fine with the desired output. Thanx for all the suggestions.

Leaving the reject link in place would also have yielded the desired result.

Time to mark this thread as resolved?