Posted: Mon Aug 19, 2013 8:14 am
You are reading data that has characters that aren't supported in the default character set. Unless you have NLS enabled for the system and the project, this isn't easily fixable.
Scratch that... This isn't easily fixable regardless!
There are several hurdles to overcome
- Determine if NLS is enabled for your system. If it isn't, you have to re-install DataStage to add it, which is NOT trivial. Adding NLS to a non-NLS system can also cause unintended issues as jobs will start reporting "I have bad characters in my data".
- Determine what character set the incoming data source is using, and see if that character set is supported on your target. If so, great. Just modify the job to use the correct NLS character set (its an option on the various stages) and the data will flow through.
- If the character set isn't supported on the target system, then you have to three options
1) Let the job replace the characters with a "?" as it is probably doing right now.
2) Scan each of the incoming text fields that contain "bad" characters and strip out the "bad" character (not recommended).
3) Scan each of the incoming text fields and substitute a supported character for each "bad" character. (Recommended)
Scanning every character in a field is quite time consuming and can really slow down a job.
Note - I also recommend looking at the "Globalization Guide" for DataStage, which covers implementing and using NLS. This is the latest version.
Scratch that... This isn't easily fixable regardless!
There are several hurdles to overcome
- Determine if NLS is enabled for your system. If it isn't, you have to re-install DataStage to add it, which is NOT trivial. Adding NLS to a non-NLS system can also cause unintended issues as jobs will start reporting "I have bad characters in my data".
- Determine what character set the incoming data source is using, and see if that character set is supported on your target. If so, great. Just modify the job to use the correct NLS character set (its an option on the various stages) and the data will flow through.
- If the character set isn't supported on the target system, then you have to three options
1) Let the job replace the characters with a "?" as it is probably doing right now.
2) Scan each of the incoming text fields that contain "bad" characters and strip out the "bad" character (not recommended).
3) Scan each of the incoming text fields and substitute a supported character for each "bad" character. (Recommended)
Scanning every character in a field is quite time consuming and can really slow down a job.
Note - I also recommend looking at the "Globalization Guide" for DataStage, which covers implementing and using NLS. This is the latest version.