New Line character on Windows

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

New Line character on Windows

Post by Bostonian »

Hi,
I am trying to build a sequencer to get a list of files from windows server and loop them in 1 by 1.

Execute Command --> Start Loop ---> Job Activity --> End Loop

In the execute command, I am executing the following command to give me the list of file names in the directory.

grep -L root #Src_file#*.txt

My problem is that neither can I give the \n as the looping delimiter nor can I convert it to something else like ';'. I searched but I am just not able to locate the correct command. Can anyone please help?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Not sure exactly what you are trying but the record delimiter on Windows is two characters: a CR/LF pair.
-craig

"You can never have too many knives" -- Logan Nine Fingers
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Rather than the grep, what about using ls -m which generates the output in a single comma delimited string?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

Post by Bostonian »

I tried ls -m but it doesn't work on Windows Power Shell or command prompt.

I simply want to loop the sequencer as many times as I have the number of files in the directory. The grep command lists the file names correctly but I am not able to give a new line character as the start loop delimiter.

e.g.
If the output of grep -L root #Src_file#*.txt gives the following output

abc.txt
xyz.txt

Then the job activity should run twice with abc.txt in the 1st run followed by xyz.txt as input.

I tried CR/LF as delimiter but its not taking it.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

In a User Variables activity change the line terminators (now @FM) to commas, so that you now have a comma-delimited list for the StartLoop activity, and remove the final comma so that you don't have an empty final name.

Code: Select all

Field(Convert(@FM, ",", ExecCommand.$Output), ",", 1, Count(ExecCommand.$Output,@FM))
Last edited by ray.wurlod on Thu Oct 09, 2014 5:37 pm, edited 1 time in total.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I just mean "CR/LF" as in a Carriage Return / Line Feed pair rather than a literal if that's what you tried. Sometimes you need the hex values or "\r\n" or who knows what. Or just play the @FM field mark game Ray posted. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

Post by Bostonian »

Ray and Chulett - Thank you. But when I use Ray's command the output is basically the number of files that the ls command is fetching. It is not giving the actual file names.

E.g. if my ls command lists

abc.txt
def.txt

Field(Convert(@FM, ",", ExecCommand.$Output), 1, Count(ExecCommand.$Output,@FM)) is giving the output as 2 and NOT abc.txt,def.txt as what I was expecting. :?
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

What command are you using?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

Post by Bostonian »

In the user variable stage,

Field(Convert(@FM, ",", ExecCommand.$Output), 1, Count(ExecCommand.$Output,@FM))

In the execute Command stage,

grep -L root #Src_file#*.txt

I also tried ls #Src_file#*.txt
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You missed the quoted comma as the second argument of the Field() function.

Code: Select all

Field(Convert(@FM, ",", ExecCommand.$Output), ",", 1, Count(ExecCommand.$Output,@FM))

                                               ^
                                               |
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

Post by Bostonian »

Darn... As always, you are right Sir Wurlod. It works fine now.

thank you very much.

By any chance do have any suggestion on how I can do the following in a sequencer?

- Run the following ls command in the execute command stage. {DONE}
ls abc_123_US_*.txt.

- If it fetches any file then convert the end of line into comma. {DONE}

- After which, split the file name into 2 additional outputs,
var1 = abc_123
var2 = US

- Pass the above 2 variables + the delimited file name(var 3 = abc_12_US.txt) into a job activity.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Look into the User Variables stage.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Generate the parameter value in the Job activity using an expression, for example

Code: Select all

UserVars.var1 : "_" : UserVars.var2 : ".txt"
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Bostonian
Premium Member
Premium Member
Posts: 33
Joined: Fri Nov 20, 2009 3:32 pm

Post by Bostonian »

But how do I populate UserVars.var1 and UserVars.var2 from the file name coming out of the start_loop_activity.counter?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Looping is discussed in a goodly amount of detail in the manuals, in your shoes I would hunt those down and study the examples there. Also, perhaps this post of mine might help or you can do an exact search for $Counter here (note the dollar sign) for many examples people have posted over the years.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply