Code: Select all
cat TestFile.txt | tr '\000' '?'
If I use a sequential file stage without a filter on that fixed width file it reads the file correctly. If I put in a filter command that merely echos the input ("cat -") it also works correctly.
But if I put in the tr command "tr '\000' '?'" as listed above it should work, but it doesn't - the null characters are stripped out of the string; and the job gives warning due to a short read on the fixed length record and those records with nulls are skipped.
If I change the command to replace all spaces with question marks (using \040 instead of \000) then the tr command replaces spaces correctly but the nulls are still automagically removed. This tells me that the problem isn't with quote characters and backslashes and 'escaping' strings in the filter stage.
The files in question are large, so I cannot easily shell out to make a cleansed temporary copy and the incoming data format is fixed. I do need to handle the nulls (and other non-displayable characters) outside of DataStage because, in addition to the tr command, I also need to remove the last trailing character using the
Code: Select all
sed '$s/.$//'
I don't think that this problem is restricted to V8 and was curious if anyone has any ideas on what else I might try. Will perl allow character substitutions?