Processing list of files in a Sequence job

[arun_im4u]

Hello,

I have to process a list of files in a loop in the sequence and I am running into an issue. First I get the list of files by doing a ls -m(Execute Command Activity) and then use the start loop activity to process the files thorugh the sequence of jobs one after the other.

The issue is the last file in the list does not pass through properly and the sequence aborts saying [ParamValue/Limitvalue is not appropriate]. I see that the output from the ls -m command has put a new line feed at the end of the last file name. I tried removing it in Ereplace and Convert, but it still gives the same issue.

Any help would be great.

[ray.wurlod]

Use a User Variables activity to remove the final line terminator from the output.

Code: Select all

Left(CmdActivity.$CommandOutput, Len(CmdActivity.$CommandOutput)-1)

This will work on both UNIX and Windows environments because DataStage has translated either kind of line terminator to field mark by the time it gets into the User Variables activity.

You might also like to convert the field mark characters (@FM) to something more amenable as the separator character for the StartLoop activity, for example semi-colon.

Code: Select all

Convert(@FM, ";", Left(CmdActivity.$CommandOutput, Len(CmdActivity.$CommandOutput)-1))

[chulett]

You're not the first to see this problem or to post about it here and yes you need to remove the last delimiter from the delimited list. Not sure how EReplace or Convert would be helpful, I seem to recall using substring to accomplish this - basically:

DelimitedList[1,length(DelimitedList) - 1]

[Kryt0n]

Convert @FM to blank or Extract the first field from the field mark delimited value will both work

[arun_im4u]

Thanks Ray. It worked. I did not realize it was putting @FM at the end. I thought it was putting a Unix new line at the end.

[ray.wurlod]

Both things are happening.

The ls command puts a newline at the end, so as to position the cursor on the next line when issued interactively.

DataStage converts all line terminators to @FM so that applications are portable across platforms.