row count check in a file, then run job

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
edmtenzing
Premium Member
Premium Member
Posts: 22
Joined: Wed Jul 23, 2008 5:59 pm
Contact:

row count check in a file, then run job

Post by edmtenzing »

Hi,

I need to check the row count for a file. If it is having more than 1 row, then i run job 1, else i run job 2. How do i do this in a job or sequencer. Thank you
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

An Execute Command stage could count the number of records in the file:

Code: Select all

wc -l <filename>
After that you just need triggers from there to two Job Activity stages, one could check for $CommandOutput=1 and the other could be simply an Otherwise trigger for when it is greater than one. Assuming the result could never be zero or a 'file not found' error, of course.
-craig

"You can never have too many knives" -- Logan Nine Fingers
edmtenzing
Premium Member
Premium Member
Posts: 22
Joined: Wed Jul 23, 2008 5:59 pm
Contact:

Post by edmtenzing »

Thanks ya. The command works, but the trigger part, still confusing to me.

what i need to do is to check whether is there any rows return in the file, if there is no rows return (row count=0) then i need to run another job.

The trigger 'expression type', which one i should use? If i use 'custom and add $CommandOutput=1, it give me the warning 'variable $commandOutput not define. What 'ReturnValue' expression means
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

$CommandOutput needs to be qualified with the name of the Execute Command activity to which it refers, for example GetFileCount.$CommandOutput
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Use the External Parameter Helper under the ellipsis (...) to pick the variables for you rather than trying to type them in.
-craig

"You can never have too many knives" -- Logan Nine Fingers
edmtenzing
Premium Member
Premium Member
Posts: 22
Joined: Wed Jul 23, 2008 5:59 pm
Contact:

Post by edmtenzing »

Thanks! I got &CommandOutput work out. When i set the &CommandOutput=1

but when i execute the command wc-1 filename. The status in the log

" Output from command ====>
0 filename "

output comes out with filename, not 0 or 1 or 2. So the trigger does not work.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Just grab the number before the filename by using this function:
Field(EC_StageName.$CommandOutput," ",1) just replace the EC_StageName with the name of your execute command stage.

Also - make sure your wc command uses a fully qualified (from root down) pathname.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
edmtenzing
Premium Member
Premium Member
Posts: 22
Joined: Wed Jul 23, 2008 5:59 pm
Contact:

Post by edmtenzing »

This does not work, i tried LEFT as well, it still return filename. I got a solution to return the number only which is

wc -l filename | awk '{print $1}'

But my trigger to run the job still does not work when i put $commandoutput=0 to run the job (my job does not run). The log shows the ouput like this

Output from command ====>
0
dxwwq557
Participant
Posts: 16
Joined: Wed Nov 10, 2010 9:19 pm
Location: Chengdu

Post by dxwwq557 »

like ur code

Code: Select all

wc -l $filename | awk '{print $1}'
maybe u want post the Filename to $filename but it can't work.
if ur define the file name to filename , it will work.
if u still want to post the filename,u can use the shell script.

Trigger Expression code like this:

Code: Select all

CountRows.$CommandOutput <> 0
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You need to handle the Field Mark that is part of the returned value. It's been posted here ad naseum but one more time...

Code: Select all

Convert(@FM,"",EC_StageName.$CommandOutput)=0
-or-

Code: Select all

EC_StageName.$CommandOutput<1>=0
The code Andy posted should have worked just fine, I'd suggest double-checking it or posting the exact syntax you used so we can.
-craig

"You can never have too many knives" -- Logan Nine Fingers
edmtenzing
Premium Member
Premium Member
Posts: 22
Joined: Wed Jul 23, 2008 5:59 pm
Contact:

Post by edmtenzing »

Thanks a lot for your guide and patient ya. It works now! I am so happy. Both command works. This is the first time i use this execute command, trigger with custom condition.
dxwwq557
Participant
Posts: 16
Joined: Wed Nov 10, 2010 9:19 pm
Location: Chengdu

Post by dxwwq557 »

there is another way to do this :

code for the command.

Code: Select all

wc -l
code for the parameter,this is the job parameter.

Code: Select all

#filename#
code for the trigger:

Code: Select all

FIELDS(CountRows.$CommandOutput," ",1)  >  0
it runs well , and u can post the file from job parameter.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Field() not Fields() and Andy already posted that particular solution. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply