DSXchange

My datastage has been acting weird of late. I'm wondering if anybody else has run into something similar.

I have a Flow that reads in numerous data tables via ODBC and puts them into their own data sets on the hard drive. I noticed that when I do a View Data now on a data set stage, a table which has 112 rows of data only gives me the first 30. I have the prompt specified for View Data to show the first 250 rows, and no skip or anything.

I took the one of these out and put it into its own flow. Same thing happened. So i replaced the data set stage with a sequential one to output to a CSV file. Same thing happened. I went to the hard drive, opened it with a text editor, and it showed 112 records.

I tried restarting the services and that did not work. Anybody run into this before?

Has anybody run into anything like this?

Only when there is a metadata mismatch between the viewing stage and the actual data. Any chance there's "something wrong" with the rows that don't show up when you compare the column definitions to the actual data?

chulett wrote:Only when there is a metadata mismatch between the viewing stage and the actual data. Any chance there's "something wrong" with the rows that don't show up when you compare the column definitions to the actual data?

I had thought of that too because it's definitely happened to me before. Here was my line of "thinking" and action I took:

1) Imported ODBC definitions again from the database.
2) Made a brand new flow with ODBC Stage (with the new definitions) --> Dataset
3) Compile/Run
4) Told me 112 rows got inserted into the Dataset
5) Checked DataStage Director and saw no warnings or errors.
6) "View Data" and only the first 30 of 112 rows showed up.
7) Made a similar flow, except the output was a sequential file instead of a dataset.
8) Compile/Run
9) Check sequential file via View Data, same result. Check it on the file system, saw 112 rows.
10) As a result, I made a new flow where the sequential file which shows 112 rows of data is input, output to another file, but has a reject link. 0 rows of data went to the reject link, and 112 rows passed.

View Data runs through an options window that allows you to limit the number of rows returned. Is yours set to 30?

ray.wurlod wrote:View Data runs through an options window that allows you to limit the number of rows returned. Is yours set to 30? ...

Nope. I set it to 250, then went way ahead and set it to 5000. Same results across the board.

Is there a Ctrl-Z character in your data? (This is the Windows end-of-file marker.)

ray.wurlod wrote:Is there a Ctrl-Z character in your data? (This is the Windows end-of-file marker.) ...

I think this ultimately ended up being the reason. Thanks. I opened the sequential file in a hex editor and noticed a 1A (in Hex) character in a field in the 30th record (and also other places).

Thanks again.

And, of course, 0x1A is Ctrl-Z.