How to check the record count before dropping the file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Shree0410
Participant
Posts: 70
Joined: Tue Nov 29, 2005 7:25 pm

How to check the record count before dropping the file

Post by Shree0410 »

Hi all,

In my current integration project, we all share a common file sharing server. Datstage drops the files for other system to pick the file after the required transformations and validations. The requirement is to drop the file from DataStage only if the record count is equal to or greater than one. Can anyone help on how to handle this in a script so I can run this script in the After-job routine.

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

In other words, a non-zero record count. :wink:

Lots of posts on this subject lately. You can do it all in a Routine or a shell script, whichever you feel more comfortable with. Search for topics discussing things like DSGetLinkInfo with the 'row count' info-type argument to check the number of records that went down the link to the Sequential File stage producing the file. Or a simple shell check of the file itself - I think it is "if -s" that is true if the file exists and has more than zero bytes in it. Or "wc -l" to count the lines, which can be helpful if there is a header record.

Many ways to skin any given cat around here.
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

What do you mean by "datastage drops the file". Do you mean creates it? If thats the case, then the file will be created regardless of the record count. If you mean something else then please clarify.
You can check for number of lines in a file by wc -l command. If it returns 0 that means the file is empty. If you have the "First line is column names" option checked, then wc -l will return 1 and that will mean the file is empty. You can easily script this. If you want more unix help, google can retrieve vast amount of info for you to get started.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Shree0410
Participant
Posts: 70
Joined: Tue Nov 29, 2005 7:25 pm

Post by Shree0410 »

DSguru2B wrote:What do you mean by "datastage drops the file". Do you mean creates it?.
Right now, after the completion of the DS job, I am using After-job routine to drop the file on the shared server.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Ok great. I bet you already have a script in place to do that. Just add small piece of code to check the line count and then drop the file.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Post Reply