Stupid Beginners Ques re lookup stage - HELP I am to dense
Posted: Thu Nov 11, 2004 5:52 pm
Hi, I'm am trying to familiarise myself with PX and have the following problem.
2 files both files 50 records (to keep it easy)
CustData (keyed by custid(int) 1-50)
CustLoookup (keyed by custid(int) 1-50)
PX job 1 load CustLookup into lookup dataset modulas partioned
on custid
PX job 2 read CUstDate -->> tfm with partion mod on cust id
lookup stage ref custlookup - reject any not found records
- out put found records
results
PX job 2 outputs 12 records and rejects 38
the 12 records are all in partion 0 (custId = 4,8,12 etc)
the 38 records are in partions 1-3 (custid = 1,2,3,5,6,7 etc)
I have a .apt config with 4 nodes
Question is ???????????????????????/
I have worked out after a bit of sleuth work , i.e. using the data above that I am not partioning correctly, but since both the lookup data set and the lookup stage are partioned by mod(custid) - I have checked this with Peek stage how come my lookup stage only seems to find data on Part0
I KNOW I am missing something but any tips / gotchas / pithy comments and abuse welcome, I have speant an evening on this when I should have been down the pub so would like to get it sorted
Thanks in advance
Fridge
2 files both files 50 records (to keep it easy)
CustData (keyed by custid(int) 1-50)
CustLoookup (keyed by custid(int) 1-50)
PX job 1 load CustLookup into lookup dataset modulas partioned
on custid
PX job 2 read CUstDate -->> tfm with partion mod on cust id
lookup stage ref custlookup - reject any not found records
- out put found records
results
PX job 2 outputs 12 records and rejects 38
the 12 records are all in partion 0 (custId = 4,8,12 etc)
the 38 records are in partions 1-3 (custid = 1,2,3,5,6,7 etc)
I have a .apt config with 4 nodes
Question is ???????????????????????/
I have worked out after a bit of sleuth work , i.e. using the data above that I am not partioning correctly, but since both the lookup data set and the lookup stage are partioned by mod(custid) - I have checked this with Peek stage how come my lookup stage only seems to find data on Part0
I KNOW I am missing something but any tips / gotchas / pithy comments and abuse welcome, I have speant an evening on this when I should have been down the pub so would like to get it sorted
Thanks in advance
Fridge