Hi,
Storing extracted data into (temporary) delimited files for subsequent processing within a DataStage job seems to be problematic when the data contains the delimiting character or embedded <CR><LF>.
I am wondering if there is a way around this problem.
Thanks Nima
temp delimited files that contains the delimited character
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Actually, if you define a quote character for your string data you can have any number of occurrences of your delimiter character (as well as your quote character).
The difficulty is with your line terminator character, the parsing algorithm cannot tell the difference between an embedded character and a true line break. If you know you will have line or record terminators in your data then the easiest solution is to define your data as fixed length and remove both quote and terminator characters -- that way DS just reads and writes <line-length> characters per record irrespective of the contents. Not necessarily a great solution if you have a varchar(1024) column that only contains 4 characters, but it does work.
The difficulty is with your line terminator character, the parsing algorithm cannot tell the difference between an embedded character and a true line break. If you know you will have line or record terminators in your data then the easiest solution is to define your data as fixed length and remove both quote and terminator characters -- that way DS just reads and writes <line-length> characters per record irrespective of the contents. Not necessarily a great solution if you have a varchar(1024) column that only contains 4 characters, but it does work.