Page 1 of 1

Inconsistent View Data in Sequential File stage

Posted: Thu Dec 17, 2009 8:41 am
by vnspn
Hi,

We have a Sequential File stage as source that reads a .csv file. This file has 115 records in it. The last few records have a value of -1 in one of the columns.

When we do a view data on this stage, the number of records that show on the screen are inconsistent. Sometimes it shows all records including the last few records with '-1' value, or sometimes it does not show the last few records that have '-1' value. I have the limit for view data as the highest 9999.

We would like to know if its a problem with just the view data or something to do with the format within the file.

Thanks.

Re: Inconsistent View Data in Sequential File stage

Posted: Thu Dec 17, 2009 9:17 am
by Sainath.Srinivasan
vnspn wrote:Sometimes it shows all records including the last few records with '-1' value, or sometimes it does not show the last few records that have '-1' value. I have the limit for view data as the highest 9999.
What is common in the 'sometimes' ?

Re: Inconsistent View Data in Sequential File stage

Posted: Thu Dec 17, 2009 9:39 am
by vnspn
I'm doing nothing differently.

I just repeatedly do a view data, and at times is shows all records and at certain times does not show the last few records with -1.

Posted: Thu Dec 17, 2009 10:07 am
by chulett
What happens when you run the job? Are those last "-1" records processed or skipped? Skipped records indicate a metadata issue. Typically.

Posted: Thu Dec 17, 2009 1:51 pm
by vnspn
No, they are not skipped. So far when we have run the job, I see them in the output. The problem is only when I view the data from the stage. Because it sometimes shows up and sometimes does not with view data, we are worried that, if that would drop those records during some runs.

Posted: Thu Dec 17, 2009 2:08 pm
by ray.wurlod
What's the data type? -1 is a valid value for any signed data type.

Posted: Thu Dec 17, 2009 4:31 pm
by vnspn
The datatype is Interger.
I'm not getting any error /warning message when I view data. But, it just misses some records...

Posted: Thu Dec 17, 2009 6:06 pm
by ray.wurlod
What are your sampling specifications when you use View Data? If you specify only 10 rows, that's all you'll get.

Posted: Thu Dec 17, 2009 6:52 pm
by chulett
If you repeatedly view the exact same static data over and over using the same settings and the results are not identical, you need to report that to the vendor - i.e. your official support provider.

Posted: Mon Dec 21, 2009 12:05 pm
by vnspn
Ray - I have my sampling specfication with the highest value possible, '9999'.

chulett - You have precisely stated our scenario. When we repeatedly view the exact same static data over and over using the same settings, the results are not identical. We shall get in touch with our vendor support as you have mentioned.

Thank you.