Handling multiple commands inside FilterCommand in Seq File?

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

sunshine
Participant
Posts: 61
Joined: Wed Jan 04, 2006 10:24 am

Handling multiple commands inside FilterCommand in Seq File?

Post by sunshine »

lets say i need to execute some commands in FilterCommand.

for eg:

tar cf mytar.tar A.txt B.txt C.txt
mv sample.html public_html
chmod go-rwx biglist

my question is:
should i write all these commands in a .txt file and give the path of this .txt file in Filter Command?

or some other solution?


Thnks
sunshine
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Those are not appropriate Filter commands. Those could go in a Command stage in a Sequence job. They could also be used directly or via a script before/after job with ExecSH or ExecDOS.
-craig

"You can never have too many knives" -- Logan Nine Fingers
sunshine
Participant
Posts: 61
Joined: Wed Jan 04, 2006 10:24 am

Post by sunshine »

ok chulett,

assume that u want to use multiple proper commands, inside filtercommand..

for eg:
cat ...
sep ....
head ....

in this case, what should i do? :)

my question is,should i need to write in .txt file and use the filepath inside FilterCommand or something else...

Thnks
sunshine
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

There's no text file support inside the Filter command, so you can't take that route. Multiple command support there is probably pretty limited. Dont' expect to be able to run multiple commands in a serial fashion there, but you may be able to pipe commands together.

Keep in mind how the Filter works. The contents of the Filename in the Sequential stage are 'redirected' into your command as Standard In and then Standard Out of your Filter command is read as input by the stage. So if you can get your command(s) working on the command line in a similar fashion, then you should be able to leverage those same commands in the Filter.

Probably still best to do all this who-ha 'before job' and create a new file that is read by the stage.
-craig

"You can never have too many knives" -- Logan Nine Fingers
sunshine
Participant
Posts: 61
Joined: Wed Jan 04, 2006 10:24 am

Post by sunshine »

sill i haven't got solution.


assume that i need to execute multiple commands
eg:
cp a1 a2
cp a2 a3
cp a3 a4

these i need to execute before executing job.

if it is only one command,
just in can write in Before Subroutine .

but in this case,what i need to do?

Lookinf forward for ur reply

thanks
sunshine
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Actually, you've gotten several solutions. Stop fixating on the 'Filter' ability of the Sequential file stage, it's not appropriate for any of the scenarios you've made up yet.

If you need to 'pre-process' a file or series of files to build something the Sequential File stage can read, then as previously noted do something 'before job'. That can literally be something done before the job runs or it can be something you do in a Before Job Subroutine.

1) Write a script to do the file manipulation:
1a) Execute the script using a Execute Command stage in a Sequence job before the Job Activity that runs your job.
1b) Execute the script using a custom routine and the Routine Activity stage in a Sequence job before the Job Activity that runs your job.
1c) Execute the script as a Before Job Subroutine using ExecSH.

2) String the commands together, separated by semi-colons:
2a) Execute the commands using a Execute Command stage in a Sequence job before the Job Activity that runs your job.
2b) Execute the commands using a custom routine and the Routine Activity stage in a Sequence job before the Job Activity that runs your job.
2c) Execute the commands as a Before Job Subroutine using ExecSH.

There's several solutions. Others can be crafted as well, but that should get you started. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Here's a friendly piece of advice. You are ignoring what has been told to you. This is a bad habit, because you are ignorant of something and Craig has nicely tried to explain it to you twice. The filter command is supposed to return SCREEN OUTPUT as streaming rows to be redirected to the internal pipe setup by the stage instead of reading the physical file.

That being said, in unix you can execute several commands by separating them with a semi-colon ";".

Code: Select all

tar cf mytar.tar A.txt B.txt C.txt ; mv sample.html public_html ;chmod go-rwx biglist 
However, now armed with this knowledge, you STILL CANNOT DO WHAT YOU HAVE ASKED with the filter feature.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
sunshine
Participant
Posts: 61
Joined: Wed Jan 04, 2006 10:24 am

Post by sunshine »

sorry guys,

i was misunderstood.

Thank you for all,for spending so much time on this topic.

Thhnks once again

sunshine
DSRajesh
Premium Member
Premium Member
Posts: 297
Joined: Mon Feb 05, 2007 10:37 pm

Post by DSRajesh »

Hi

How can we give compress command in filter command of sequential file

when i am trying to give gunzip May07.GZ ,its throwing error as No such file or directory .

but physically file exists.

Can you please help me
RD
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The file physically exists - where? Use an absolute path unless your file actually does exist within the job's project, which is where you've told it to look for it.
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSRajesh
Premium Member
Premium Member
Posts: 297
Joined: Mon Feb 05, 2007 10:37 pm

Post by DSRajesh »

Hi Chullet,

Approaching the same.
But still getting that error.
Pls help me in that regard.

Regards
Rajesh Deva
RD
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The filter needs a data stream (stdin) to work with. I'm not at DS session now to check, but it should work with "gunzip -" or, failing that, use an external command stage to perform the compression or decompression.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Make sure you have the full path to the file mentioned and that you've told gunzip to 'write on standard output'. Make sure it works from the command line and writes the uncompressed data to 'the screen' and then it will work in the Filter.
-craig

"You can never have too many knives" -- Logan Nine Fingers
DSRajesh
Premium Member
Premium Member
Posts: 297
Joined: Mon Feb 05, 2007 10:37 pm

Post by DSRajesh »

Hi

Is the format of .gz file is in binary format i am unaware of exeact format of gz file
plesae help me here.
RD
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The format of a zipped file is binary. Various algorithms including Lempel-Ziv-Welch are used that take advantage of the fact that different byte or group of byte combinations occur with varying frequencies within a file and that not all 256 possible values in a byte are used throughout the file (huffman encoding). This allows files to be compressed losslessly.

When reading or writing a compressed file I will use an external command stage and use the command "gunzip {filepath/name} -" or "uncompress {filepath/name} -" to read a compressed file and stream the data into the job.
Post Reply