Line Terminators

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
phantom99
Participant
Posts: 2
Joined: Thu Mar 25, 2004 11:35 am

Line Terminators

Post by phantom99 »

Hi ...

We are receiving mainframe files, sent to our unix server comma delimited, and with "'s as our quote characters. We have noticed that within the data, in some rare circumstances there are line terminators. We have dealt with this issue by setting the line terminators property to 'yes' in the source sequential file stage on the suspect column. This works, and DS as a result is able to read the data properly.

The problem is, after doing some transformations etc, we land the data into sequential files to be used by DB2 Import. DB2 Import fails, as the data is still containing these line terminators which obviously creates broken lines.

Does anyone know of a solution to this problem ... we basically want DS to eliminate this Line Terminator as it writes out the data to the target sequential file ?

We figure we are stuck coming up with a genius routine.

Thanks a lot ...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You need to remove the terminators before you land the data. Probably the simplest way is to use the Ereplace function to either turn them into spaces or remove them all together. Depending on if you are removing LFs or CR/LF pairs (or whatever), you could do it a couple of different ways:

Code: Select all

EReplace(TheField,CHAR(10)," ",-1,0)
EReplace(TheField,CHAR(13):CHAR(10)," ",-1,0)
The first one replaces line feeds with a space while the second replaces every CR/LF pair with a single space. Close the quotes to remove the characters, essentially replacing them with nothing.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Single character replacements are more efficiently handled using the Convert function rather than Ereplace.
Also for efficiency, initialize a stage variable (or a constant in code, using Equate) so that Char(10) is only evaluated once.

Code: Select all

Convert(svLF," ",TheField)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
sivap
Participant
Posts: 14
Joined: Tue Oct 26, 2004 8:13 am

CR/LF Problem

Post by sivap »

ray.wurlod wrote:Single character replacements are more efficiently handled using the Convert function rather than Ereplace.
Also for efficiency, initialize a stage variable (or a constant in code, using Equate) so that Char(10) is only evaluated once.

Code: Select all

Convert(svLF," ",TheField)
I am sending data from SQL server to Flat file, I am facing problem with line feed . I am using this function Convert(svLF,"", DSLink3.address2). Is this syntax is correct? I transformer it showing error.
sjacobk
Participant
Posts: 9
Joined: Fri Apr 15, 2005 4:32 am
Location: India

Re: CR/LF Problem

Post by sjacobk »

The following OConv conversion will replace all control characters by a "."

OConv(The Field,"MCP")

Smitha
Smitha Jacob
Post Reply