Issue Reading in a Sequential File with trailing blanks
Posted: Tue Nov 19, 2013 12:58 pm
We are having issues reading in Sequential Files with blank values in the final columns.
For example, lets say I have a Sequential File with 5 columns:
Col_1 CHAR(5)
Col_2 CHAR(5)
Col_3 CHAR(1)
Col_4 CHAR(2)
Col_5 CHAR(6)
If my record has values in Col_1, Col_2, and Col_3, but Col_4 and Col_5 are empty, DataStage is not reading in the record properly.
If I specify the Sequential File as follows:
Record delimiter = UNIX newline
Record length = fixed
I get the following error:
FileIn,0: Bad record delimiter after fixed-length record: expected "
", got " "
It is reading beyond the UNIX newline and into the next record.
When I examine my dataset with a Hex viewer, the UNIX newline is where it's supposed to be.
If I change the Sequential File to specify, only the Record delimiter (and remove the Record length specification), I get the following error:
FileIn,0: Input buffer overrun at field "FE_PRINT_ALTERNATE_SW", at offset: 103
That field is the first blank field of the final three fields and the offset is the first byte of that field.
When I look at the record with the Hex viewer, this observation is confirmed. Starting from that field on, the remaining fields are blank.
If I edit the Sequential File and add a final byte filled with "X" (a dummy field), DataStage reads in the file and processes the data.
My problem is that we are in the midst of a conversion and we have hundreds of sequential files created by outside programmers that potentially have blank values at the ends of some of the incoming records. We would like to be able to read in the Sequential Files without having to artificially add a dummy field to the end of them.
Does anyone have any suggestions?
For example, lets say I have a Sequential File with 5 columns:
Col_1 CHAR(5)
Col_2 CHAR(5)
Col_3 CHAR(1)
Col_4 CHAR(2)
Col_5 CHAR(6)
If my record has values in Col_1, Col_2, and Col_3, but Col_4 and Col_5 are empty, DataStage is not reading in the record properly.
If I specify the Sequential File as follows:
Record delimiter = UNIX newline
Record length = fixed
I get the following error:
FileIn,0: Bad record delimiter after fixed-length record: expected "
", got " "
It is reading beyond the UNIX newline and into the next record.
When I examine my dataset with a Hex viewer, the UNIX newline is where it's supposed to be.
If I change the Sequential File to specify, only the Record delimiter (and remove the Record length specification), I get the following error:
FileIn,0: Input buffer overrun at field "FE_PRINT_ALTERNATE_SW", at offset: 103
That field is the first blank field of the final three fields and the offset is the first byte of that field.
When I look at the record with the Hex viewer, this observation is confirmed. Starting from that field on, the remaining fields are blank.
If I edit the Sequential File and add a final byte filled with "X" (a dummy field), DataStage reads in the file and processes the data.
My problem is that we are in the midst of a conversion and we have hundreds of sequential files created by outside programmers that potentially have blank values at the ends of some of the incoming records. We would like to be able to read in the Sequential Files without having to artificially add a dummy field to the end of them.
Does anyone have any suggestions?