concatatination of 3 files

[rumu]

Hi,
My requirment is in a job I have to concatenate 3 files (different date) and to use it as source file and load that resulting file into target stage(sequential file).
wat i did in my job,in a sequential stage i checked "filter commands" option then in commmand text box give-cat a b c>d----command whre a,b,c are three files need to be concatenated in to a single file say d(in my example).
then in file text box i use dirpath/d
job is getting aborted giving error------unable to open file d in dirpath.
i know there is another way out of using custom routine to execute unix commnd but any one can help me why my job is getting aborted?
Thanks in advance

[elavenil]

It seeems to me, the concatenation of those three files did not happen. So ensure that your cat operation is successful so that file d can be used as input.

Is there any error/warning messages in the log?

Regards
Saravanan

[rumu]

It seeems to me, the concatenation of those three files did not happen. So ensure that your cat operation is successful so that file d can be used as input.

Is there any error/warning messages in the log?

Regards
Saravanan

Thanks for ur response.No in director there is no error msg regarding concatenation-it simply shows "can't open d in dirpath"

[kumar_s]

What happen when u try the same command in unix comand prompt??

regards
kumar

[chulett]

The Filter command is not meant to work that way. It is expecting the actual filter command to be sending output to Standard Out and it attempts to source from that.

For concatenation, put what you have in the Filter command in a Before Job ExecSH call and leave the Filename in the Sequential stage picking up the concatenated filename.

[ray.wurlod]

Don't run the command cat a b c > d as a filter. Instead, run it as a before-stage subroutine.

As Craig, said, filters don't work that way. If you want to use a filter, the command would be cat a b c and DataStage would use stdout from the cat command as its source of rows.

[rumu]

Don't run the command cat a b c > d as a filter. Instead, run it as a before-stage subroutine.

As Craig, said, filters don't work that way. If you want to use a filter, the command would be cat a b c and DataStage would use stdout from the cat command as its source of rows.

Hi Craig
I have used this following command in inputvalue for ExecSh beforjobsubroutine --cat a b c>d
used dirpath/d in stg file name
job is running successfully and getting reqrd result,but in director it is shown that in event under info type--
ConCatJob..BeforeJob (ExecSH): Executed command: cat a b c>/d
*** No output from command ***
Why it is being shown that *** No output from command ***
when file d is being created in specified dir?
Thnx craig and ruy for your valuable suggestion

[chulett]

Why it is being shown that *** No output from command *** when file d is being created in specified dir?

It just means that running your command did not output any additional lines to the screen. What happens when you run this from the command line yourself? It simply returns to the prompt without a message.

If there were any error or status messages sent to the screen, they would be captured in that log message.

[rumu]

Why it is being shown that *** No output from command *** when file d is being created in specified dir?

It just means that running your command did not output any additional lines to the screen. What happens when you run this from the command line yourself? It simply returns to the prompt without a message.

If there were any error or status messages sent to the screen, they would be captured in that log message.

Thnx Craig,
Can u explain me how does this filter command work?I mean when to use them?If i browse for filter commnd-list shows D_DS_JOBS ...etc,cant we use unix command over here?say if i want to sort my input file i need to execute BeforeJobsubroutine---------ExecSh and in input value "sort" command need to be given?why not in filter command?

[chulett]

Ray and I have both given an explanation of how the Filter command works. You can also check the online help by pressing the 'Help' button that's inside the Sequential File stage and reading what it says there.

In a nutshell, it expects a command that - when applied against the file in the Filename field - will produce a stream of information to Standard Out. That is what the stage then 'reads' as input.

Or you can get away with putting the entire command into the Filter. For example, one way to do your concatenation in a virtual manner without actually creating a concatenated result file would be to follow Ray's advice and put cat a b c in the Filter command and /dev/null in the Filename.