Hi All,
We have 2 jobs. In Job1 a dataset is created. In job2, dataset is read and then used for join.
There are 1058 records loading into dataset in job1. When i run job 2, only 171 records are being read.
According to log, I think data is getting rejected. I see something like this below
repartition(0),1: Input 0 consumed 271 records
repartition(0),1: Output 0 producecd 271 records
I do not see any reason why data should be rejected.
Your help is appreciated.
Regards,
Vijay
Problems reading from a Dataset
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 138
- Joined: Wed Jul 16, 2008 9:51 pm
- Location: Kolkata
-
- Participant
- Posts: 11
- Joined: Thu Feb 10, 2011 1:09 am
- Location: Kuala Lumpur
-
- Premium Member
- Posts: 730
- Joined: Tue Nov 04, 2008 10:14 am
- Location: Bangalore
Use dataset management utility to find out how many records are loaded into the datastet.
If the dataset has the number of records that are loaded by job 1 then the problem is with job2 else the problem is with job1.
Even if the configuration file is not the same Datastage would repartion the data appropriately while reading it in job2.
If the dataset has the number of records that are loaded by job 1 then the problem is with job2 else the problem is with job1.
Even if the configuration file is not the same Datastage would repartion the data appropriately while reading it in job2.
-
- Participant
- Posts: 6
- Joined: Thu Jan 20, 2011 11:09 pm