Low values in Mainframe file

[priyadharsini]

Is it possible to identify low values in complex flat stage (source stage)?

[ray.wurlod]

Only if you know how "low values" is defined/implemented. For example it will be different for different data types.

[kduke]

If you are coming from a mainframe then low values a lot of times is a char(0). This can mess up a lot stuff. Remember PX jobs are really C++ underneath covers. char(0) can trigger an end of string. So functions like len() and other functions may give funny results. You need to pick the right charset and you need probably use convert to strip out char(0).

[FranklinE]

Assuming that your source system is in Cobol -- that being where the term "low values" is normally used -- the definition of each field is your correct starting point. Low values is the filling of x"00" in each byte. The difficulties start because x"00" is the ASCII code for null.

The safest way to handle this is to import your table definitions from the Cobol copybooks. They should be text files that show the field definitions using Cobol syntax. There is also the development of the Cobol programs and how well the hold to coding standards, one of which is very simple: Character fields -- PIC X -- must be initialized or filled with spaces, not low values.

If the Cobol standards are not being met, you must inspect your character fields and replace low values with spaces. Again, this is the best practice, because with variable length fields being the exception rather than the rule you will need to maintain field and record lengths or you will have problems every step afterwards.

The FAQ Using Mainframe Source Data at viewtopic.php?t=143596 provides a few more details to consider.

kduke: The charset is not critical. It's understanding how the mainframe charset translates to the one required on the DataStage side. Changing charsets should be a last resort, in my opinion.

[priyadharsini]

Thank you all for your replies. At mainframe side the initialization is not done. Convert function is used to convert the low values to spaces.