Missing Records when using Join Stage with Datasets
Moderators: chulett, rschirm, roy
Missing Records when using Join Stage with Datasets
I have a Dataset which contains approximately 2.0 million records. When I connect the dataset to a Join stage only 1.3 million of the records are being read. I can put a Transformer or Copy stage between the Dataset and the Join and all records from the Dataset will be read. Any ideas on why connection directly to a Join would cause records to be omitted? I am running version 8.5.
Thanks
Thanks
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 437
- Joined: Fri Oct 21, 2005 10:00 pm
Re: Missing Records when using Join Stage with Datasets
Data in one link and not in the other.mgray412 wrote: Any ideas on why connection directly to a Join would cause records to be omitted
Improper sorting
Improper partitioning
Keith Williams
keith@peacefieldinc.com
keith@peacefieldinc.com
-
- Participant
- Posts: 437
- Joined: Fri Oct 15, 2004 6:13 am
- Location: Pune, India
Did you check the DS record count? Try using the $ORCHADMIN ll and check it.
Also how was the DS created - using a job or $ORCHADMIN cp. Once I had used $ORCHADMIN cp and the newly created DS has a similar issue i.e. records read were less without any warning. There was some issue with char encoding while writing to new. I recreated the DS and it worked.
Also how was the DS created - using a job or $ORCHADMIN cp. Once I had used $ORCHADMIN cp and the newly created DS has a similar issue i.e. records read were less without any warning. There was some issue with char encoding while writing to new. I recreated the DS and it worked.
Regards,
S. Kirtikumar.
S. Kirtikumar.
Re: Missing Records when using Join Stage with Datasets
Thanks for your responses. Here are the answers to some of the questions that were asked:
1) Confirmed dataset record count using the Data Set Management tool in Designer
2) Using a Left Outer Join
3) Dataset was created using a DataStage job
Thanks
1) Confirmed dataset record count using the Data Set Management tool in Designer
2) Using a Left Outer Join
3) Dataset was created using a DataStage job
Thanks