Hi,
We are working in a DataStage Grid environment with multiple compute nodes. We came across a situation that worked unexpectedly.
We run a job using 2 compute nodes and 2 partitions each. The job works as expected. The lookup to an Oracle table fetches correct reference data.
When we run the same same job with 5 compute nodes, the lookup returns null value for all the incoming records.
The Lookup stage has both the input links Hash partitioned on the key column.
Could anyone suggest what could cause a lookup not to work when increasing the compute nodes.
Thanks.
Lookup in a Grid environment
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
lstsaur, I checked the log and it does not show this information. The reference table has 5 records and I see the same count that shows on the link using performance statistics. I'm sure the records are not being dropped.
ray, the partitioning algorithm on the reference link is Hash partition on the key column used for join. The stage that supplies records to the reference link is an Oracle Enterprise stage.
ray, the partitioning algorithm on the reference link is Hash partition on the key column used for join. The stage that supplies records to the reference link is an Oracle Enterprise stage.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What happens if you use Entire as the partitioning algorithm on the reference input link? I'm asking as a diagnostic question, not necessarily with a view to recommending this as the solution.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.