Executing scripts
Moderators: chulett, rschirm, roy
Executing scripts
Hi,
I have a requirement like below and iam not able to proceed any further. can anybody let me know if there is a way to do the below task.
I have 5 Input files each of different metadata. I want to get the count of each file and write into a new file. I have created a script using wc -l and tried executing in DS sequencer (by calling through script) but it says invalid command. The same command is getting executed when i use the wc -l in execute command explicitly in Sequencer, but the output is not as expected.
Example:
Input:
Input file 1: Has 1 records
Input file 2: Has 2 records
Input file 3: Has 3 records
Input file 4: Has 4 records
Input file 5: Has 5 records
Output:
NewFile: Outputfilename.txt (should contains the value as below)
File1=1
File2=2
File3=3
File4=4
File5=5
I want this to be handled in script itself and i need to trigger the script in sequencer.
Thanks in advance.
I have a requirement like below and iam not able to proceed any further. can anybody let me know if there is a way to do the below task.
I have 5 Input files each of different metadata. I want to get the count of each file and write into a new file. I have created a script using wc -l and tried executing in DS sequencer (by calling through script) but it says invalid command. The same command is getting executed when i use the wc -l in execute command explicitly in Sequencer, but the output is not as expected.
Example:
Input:
Input file 1: Has 1 records
Input file 2: Has 2 records
Input file 3: Has 3 records
Input file 4: Has 4 records
Input file 5: Has 5 records
Output:
NewFile: Outputfilename.txt (should contains the value as below)
File1=1
File2=2
File3=3
File4=4
File5=5
I want this to be handled in script itself and i need to trigger the script in sequencer.
Thanks in advance.
Gopinath
-
- Participant
- Posts: 342
- Joined: Tue Nov 04, 2008 10:38 am
- Location: Chennai, India
Yes i can execute my script outside datastage. I am giving my script name in my user variblae in sequencer and using execute stage iam passing the parameters. The script is executing with all other commands but wc -l is giving the error like "Invalid command" but if i hardcode the same command in the execute command stage then its working.vinothkumar wrote:How did you call the script in DataStage. Are you able to run your script and get the results outside of DataStage.
Gopinath
-
- Participant
- Posts: 342
- Joined: Tue Nov 04, 2008 10:38 am
- Location: Chennai, India
Input:chulett wrote:Give the full path to 'wc' in the script. If you still have problems, post the script and the exact, unedited error message(s) from the log.
wc -l TargetFilePath/FileName > Countfilepath/CountFileName
Error:
User logged in.
Type set to A.
?Invalid command
You have transferred 0 bytes in 0 files.
And getting aborted
Alternative Option tried:
Gave the wc -l command inside the sequencer using execute stage
Input:
wc -l in command line
Countfilepath/CountFileName in the parameter
Output:
Executed: wc -l ; Countfilepath/CountFileName
Reply=126
Output from command ====>
0
SH: /opt/datastage/staging/rdsunit/control/NEWTEST.txt: 0403-006 Execute permission denied.
But when i hardcode the path, getting the output like below,
" count countpath/filename.txt"
where as getting permission denied if i parameterised it.
The script is created with my ID and it has 777 option. The same script is getting executed if i removed wc command and have some other command in the script.
I am using v7.5, let me know if there is some other alternative.
Thanks a lot.
Gopinath
**** This first error is from an FTP session. FTP does not include a "wc
" option.
Newtest.txt requires a "chmod +x" to execute
Does your Datastage job attempt to do a rsh to a secondary system where the script resides. The second test is a basic error in permissions. The file could not have permissions of 777
" option.
Newtest.txt requires a "chmod +x" to execute
Does your Datastage job attempt to do a rsh to a secondary system where the script resides. The second test is a basic error in permissions. The file could not have permissions of 777
daignault wrote:**** This first error is from an FTP session. FTP does not include a "wc
" option.
Newtest.txt requires a "chmod +x" to execute
Does your Datastage job attempt to do a rsh to a secondary system where the script resides. The second test is a basic error in permissions. The file could not have permissions of 777
Can you please explain more, i dont understnad this point FTP does not include a "wc
" option.
AND
Does your Datastage job attempt to do a rsh to a secondary system where the script resides.
The scrpit has 777 option, the same script is getting executed when i run it with different command.
Gopinath
Post. The. Script.
Not one line from it or what you are putting in the stage but the entire contents of this "script" you are allegedly running. You have several problems here. Ray mentioned "ftp" because of this:
Error:
User logged in.
Type set to A.
?Invalid command
You have transferred 0 bytes in 0 files.
Those messages come from an ftp session, and you cannot execute "wc" within one without shelling out from it to the command line - hence the "Invalid command" message.
His next comment comes from the fact that, instead of redirection, for some reason you have a semicolon between your two commands:
Executed: wc -l ; Countfilepath/CountFileName
So first it does the "wc -l" without any other arguments and then tries to execute "Countfilepath/CountFileName" leading to the "chmod" comment.
Help us help you and post your complete script and explain what exactly it is it should be doing. Or if it is not in fact a script, make that clear. Help us understand what exactly it is you are doing as right now it is not clear at all.
Not one line from it or what you are putting in the stage but the entire contents of this "script" you are allegedly running. You have several problems here. Ray mentioned "ftp" because of this:
Error:
User logged in.
Type set to A.
?Invalid command
You have transferred 0 bytes in 0 files.
Those messages come from an ftp session, and you cannot execute "wc" within one without shelling out from it to the command line - hence the "Invalid command" message.
His next comment comes from the fact that, instead of redirection, for some reason you have a semicolon between your two commands:
Executed: wc -l ; Countfilepath/CountFileName
So first it does the "wc -l" without any other arguments and then tries to execute "Countfilepath/CountFileName" leading to the "chmod" comment.
Help us help you and post your complete script and explain what exactly it is it should be doing. Or if it is not in fact a script, make that clear. Help us understand what exactly it is you are doing as right now it is not clear at all.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Guessing from the flow of your logic, it appears you are ftp-ing a file from some server and checking the record count in the transmitted data.
So a successful ftp is essential for your flow.
Also chmod of your existing file will not be of much use as you may overwrite with the incoming file.
First fix the ftp issue.
Next check whether you are having a semi-colon after 'wc -l' and remove it (if present).
So a successful ftp is essential for your flow.
Also chmod of your existing file will not be of much use as you may overwrite with the incoming file.
First fix the ftp issue.
Next check whether you are having a semi-colon after 'wc -l' and remove it (if present).
-
- Participant
- Posts: 57
- Joined: Wed Oct 21, 2009 4:46 am
- Location: India