Sparse lookup gives more number of records

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Sparse lookup gives more number of records

Post by kennyapril »

I am using sparce lookup with oracle enterprise stage, it gives more
number of records than expected.I tried a sample job with stream 20 records
and reference 200 records and the output I get is 2000 records.
The two key columns have been selected as keys in the input of lookup.

please suggest if any changes are required.




Thanks
Regards,
Kenny
akonda
Participant
Posts: 97
Joined: Wed Feb 28, 2007 6:15 am

Re: Sparse lookup gives more number of records

Post by akonda »

We should use where clause in sparse lookup.

ex: Time_key = ORCHESTRATE.Timekey




Thanks
arun
arun
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Post by kennyapril »

Thanks

Do we need to use that in the query of reference table for the key column from DB stage?
Regards,
Kenny
akonda
Participant
Posts: 97
Joined: Wed Feb 28, 2007 6:15 am

Post by akonda »

correct.
arun
kennyapril
Participant
Posts: 248
Joined: Fri Jul 30, 2010 9:04 am

Post by kennyapril »

In the simple job which I said the stream has 100 records and the reference has 10000 records

I changed the query in the reference using where per_id=orchestrate.per_id

and when I run the job I receive an error

main_program: (aptoci.C:456). Message: ORA-00904: "ORCHESTRATE"."PER_ID": invalid identifier


please suggest
Regards,
Kenny
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

The error is pretty clear: There is no column named PER_ID in your orchestrate datastream (i.e. your main input link to the lookup stage).

Regarding your original question, it sounds as if there are duplicates in your reference data.

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
Post Reply