DataStage Generating Partially Incorrect OSH

[marothisu]

Hi,

This problem is for DataStage v8.1 on Windows. We have randomly run into a problem for whcih nobody can explain within the last day. I have really boiled it down to DataStage generating some incorrect OSH.

For the last 1.5 years, I have been reading in text files in sequential stages with double quote, comma delimited quite easily in numerous different flows and have never had a problem importing records as long as the column definitions were correct.

Most recently, it started to fail for no reason, even on jobs that worked 3 months ago whose input files and flows have not been changed since then. It seems as if Datastage is completely ignoring double quotes now for no reason.

When I go "View Data" in the Sequential File stage, the error that shows up is "Has import error and no default value" which makes no sense considering this flow worked perfectly fine 3 months ago and absolutely nothing with the input file or flow has been changed since then.

I checked the column definitions and everything is fine. The reason why I know everything is fine is because I replaced every comma which delimits a field with a | and it works completely fine. The length of the field is also ample enough (length of 128 while the string is only 30 characters long).

The weird thing is that if I take the fields that actually do have double quote/comma for them, and get rid of the commas, everything works fine as expected, BUT the double quote shows up in the output if I do a "View Data" in the Sequential File stage.

I have also restarted my entire machine that holds the server component and that did not work. I am not THAT familiar with OSH Script, but when doing a "View Data", I clicked on Show OSH and noticed that some fields in the definitions have quote=none instead of quote=double. For example this is part of what I see:


-schema record
{final_delim=end, delim=',', quote=double}
(
field1:string[max=19] {quote=none};
field2:int64;
field3:nullable string[max=128] {quote=none};
)


When I right click on the file as well in an Open dialog box and go to Generated OSH, it shows the same thing. I should mention this is after I compile the entire job as well.

I tracked down the OSH file on the file system and changed those quote=none to quote=double. I then ran the stage again without compiling (because the last time it was successful, just had reject output) and it worked.

Any reason why the OSH Script for this is being incorrectly generated? I am pretty stumped.

Any ideas??

Thanks,
Mike

[marothisu]

We went and forced the delimiter at the column level, but still no idea why it is/was generating overall OSH like that when we never set it at the column level in the first place.