File pattern warning in Sequential file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
snt_ds
Premium Member
Premium Member
Posts: 280
Joined: Wed Oct 18, 2006 11:53 am
Location: Pune, India

File pattern warning in Sequential file

Post by snt_ds »

We are using file pattern in sequential file for concat all files start with "aaam_classif*"and write into one output file. It is working fine if files are available, but if not found any single file that time we want job should be success.

But we are getting one warning" Couldn't find any files on host with pattern"

Could you please suggest any one how to rectify that warning?


Thanks & Regards
SNT_DS
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Make sure in advance (in the job sequence) that there is at least one file that matches the pattern. For example, use an Execute Command activity to ls the directory with the file pattern, and test its exit status or output using the Trigger expression.

You could, as an alternative, create a message handler to demote the warning message to informational.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
sudharma
Participant
Posts: 55
Joined: Wed Jan 24, 2007 10:28 am

Post by sudharma »

You can also use a wait for file activity and specify some time so that the sequence can wait for the file to appear in some specific path or location.

can you tell me clearly how your job has to run?? are the files polled on a regular basis ??

thanks
Sudharma
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Wait For File stage does not work with a file pattern.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Or you can always have an empty file say "aaam_classif_empty.txt" to ensure that your job finishes successfully. This will eliminate any extra job or stages.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Post by vijayrc »

DSguru2B wrote:Or you can always have an empty file say "aaam_classif_empty.txt" to ensure that your job finishes successfully. This will eliminate any extra job or stages.
You just beat me in posting the reply :)
Yes having a dummy-empty file always in that folder would solve your issue.
-Vijay
snt_ds
Premium Member
Premium Member
Posts: 280
Joined: Wed Oct 18, 2006 11:53 am
Location: Pune, India

Post by snt_ds »

ray.wurlod wrote:Make sure in advance (in the job sequence) that there is at least one file that matches the pattern. For example, use an Execute Command activity to ls the directory with the file pattern, and test its exit status or output using the Trigger expression.

You could, as an alternative, create a message handler to demote the warning message to informational.
Hi Ray,

First thanks to you

I have created Sequence job with Execute command:


Success
Execute activity ---------------------> Job Activity
|
|
|Fail
|
Notification activity


In execute Command:

Command: sh -c
Parameter: test -r aaam_classif*


I passed above values in execute Command. But my job is getting one warning
"s_mdf_RD..JobControl (@Execute_Command_0): Command sh -c did not finish OK, reply = '1' "

Could you please suggest me what is wrong in execute Command?
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Try the following in execute command stage to check if any files with a particular patter exist.

Code: Select all

ls <<filedir>> | grep aaam_classif  | wc -l
where filedir is the directory where the files are.
You can run that in execute command stage. If the result is greater than 0, process your job, else send out notification.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

What triggers are on the Execute Command activity? The test command returns 1 if the test condition is not met. A pipeline such as that suggested by DSguru2B may be better, as the terminating command (wc) should always set its exit status to 0. You then need to test the command output (not its status) in your triggers.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
snt_ds
Premium Member
Premium Member
Posts: 280
Joined: Wed Oct 18, 2006 11:53 am
Location: Pune, India

Post by snt_ds »

ray.wurlod wrote:What triggers are on the Execute Command activity? The test command returns 1 if the test condition is not met. A pipeline such as that suggested by DSguru2B may be better, as the terminating command (wc) should always set its exit status to 0. You then need to test the command output (not its status) in your triggers.

Thanks ray..
Post Reply