processing the file based on the string in the file.

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

processing the file based on the string in the file.

Post by dodda »

Hi all ,

Iam bit newbie to the shell scripting. And my requirement is as folows

Iam having txt file and it contains data as :::

HEADER_FILE;SPD;TCE;20080417 11:20

118982_101;20RTG STD;GRE;118982_101;ATK;1;1;01012008 00:00;01012100 00:00;1;1;STD;0;
111159_101;20RTG STD;GRE;111159_101;ATK;1;1;01012008 00:00;01012100 00:00;1;1;STD;0;
036964_101;20RTG STD;GRE;036964_101;ATK;1;1;01012008 00:00;01012100 00:00;1;1;STD;0;
015402_101;20RTG STD;GRE;015402_101;ATK;1;1;01012008 00:00;01012100 00:00;1;1;STD;0;

FOOTER_FILE;4

I want to search for a string FOOTER_FILE in the last line of the line. If the string exists then i need to process the file. Else the file need not to be processed.


Can you share the piece of code for the above requirement.

Your help is greatly appreciated.

Thanks in advance.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The simplest way would be to check for the footer from a Job Sequence and then decide whether or not to call up the processing job.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Is this in an attempt to ensure a file, which is being transferred to your system via ftp, is complete?

Me, I'd grep for the footer record and count the result:

Code: Select all

grep FOOTER_FILE <filename> | wc -l
0 = not there, 1 = there. Easy to make a decision from there.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Adding to Craig's post to make it bulletproof:

Code: Select all

tail -1 <filename> | grep FOOTER_FILE | wc -l
if that string could show up elsewhere in the file
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Of course. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Post by dodda »

I wrote the following piece of code :::

#!/bin/ksh
export count
count =` grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l`
echo $count

But it is throwing an error.
count not found.

Can anyone correct the error.
iDomz
Participant
Posts: 81
Joined: Wed Jul 25, 2007 5:25 am
Location: London

Post by iDomz »

Remove export
If you have not declared count before, no point in exporting it,right?
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Post by dodda »

Eventhough i removed the export before it is throwing the same error

footer.ksh[3]: count: not found


Any help....
iDomz
Participant
Posts: 81
Joined: Wed Jul 25, 2007 5:25 am
Location: London

Post by iDomz »

Try this on command line

count=` grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l`

No spaces on either side of =
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Post by dodda »

i tried in the command prompt as


>= `grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l`

It throws an error. ksh not found.

but i tried as
> grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l

It results 1. without blackquote it's is working fine.
iDomz
Participant
Posts: 81
Joined: Wed Jul 25, 2007 5:25 am
Location: London

Post by iDomz »

There should be no spaces between the VARIABLE (count) , the assignment operator ( = ) and the operand (`grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l`). I am not sure if it is copy-paste or phpBB, but I can see spaces in both posts.

Copy this line and execute

count=`grep -i "FOOTER_FILE" /home/user/dsstage/test/Input.txt|wc -l`

then execute

echo $count

Ping back if it still throws error
dodda
Premium Member
Premium Member
Posts: 244
Joined: Tue May 29, 2007 11:31 am

Post by dodda »

Thanks for your immediate reply.Thanks a lot.

It's working fine now. This solves my problem.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Then please mark the topic as Resolved.

ps. You don't need the quotes around FOOTER_FILE. Don't hurt but they aren't needed. And the "-i" means "ignore case", something I wouldn't do in this case.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply