Shell Script to Move a file to a Mailbox

[Developer9]

Hi All,

I have a current Process that runs "windows script " for the file movement that needs to changed to a "DataStage Process (Using shell script )"

Source :Text file is getting generated as part of Datastage Jobs processes and resides in a shared drive .

Target :ftp hostname "xxx.yyy.com " folder "ABC" is a mailbox location

Current Process :

Currently It is using following windows script to move the file

open xxx.yyy.com
user name
password
prompt
put "\\psoft8\output\FILE.TXT"%ABC%FILE
quit

What is the code I need to use it in shell script to move the file from source location to Target FTP folder mail box location ?

Please let me know any suggestions to implement this job design/ process

Thank you in advance

[PaulVL]

Google "FTP Unix Shell Script"

You can also use the FTP stage to move that file.

Secure FTP would be best.

[chulett]

Also note that what you posted is not a script but rather the response file (probably leveraged via a 'here document' approach) for the FTP session itself. Any script, such as you'll see when you do the search Paul mentioned, will need to include the actual FTP command itself. Also realize that to leverage 'secure' ftp/scp in an automated fashion, you'll need to (at a very high level) establish RSA keys between the two servers. Found this article on the subject, there's plenty more out there if you want to go the secure (password-less) route.

FWIW, I tended to avoid the use of the FTP stage since it is metadata driven. Command line ftp/scp just sends streams of bytes without much caring what kind of structure they are in.

[Developer9]

@PaulVL

Thanks for the response.

Option 1 :I am testing with the FTP Stage gave me following error

Code: Select all

FTP_Enterprise: processing failed for output connection 1: exit status = 255. FTP_Enterprise

I checked the ftp connection with the given credentials using windows command prompt

Code: Select all

ftp xxx.yyy.com
connected to xxx.yyy.com
user name 
password
ftp >ls
port command successful.
Opening ASCII Mode data connection for transfer
Mailbox ID  St  APRF    SNRF Service Ref #
ABC           Y    ZZZZ  File       xxxxxxx

Transfer complete
ftp: 1044 bytes received in 0.04 seconds 

Option 2:FTP Unix shell script (Secure ftp connection) ..I am in the process of testing.

@ Chulett

I am testing the ftp/scp commands ,for the usage in Execute command stage in Sequence jobs.


I will update the post if any progress
Thanks for the time taking in replying with your comments

[chulett]

I would suggest you write your script and execute the script using the Execute Command stage, this rather than putting the ftp commands directly in the stage.

[Developer9]

@Thank you Chulett

I am not sure how a Mail box location is different from a ftp server .

But with the current process , the customer is using windows script for placing the file in ftp mail box location

Code: Select all

put "\\psoft8\output\FILE.TXT"%ABC%FILE 

From the windows command prompt ,I am able to connect to FTP site (xxx.yyy.com) but not with DataStage FTP enterprise stage

Is there a way datastage server connect to Mail box location using FTP commands or calling the routine works ?

Please have your comments

Thank you

[chulett]

Me, I wouldn't bother with the FTP stage for something like this. Get a script working from the DataStage server and use that instead. Which means first making sure you can do it manually from the ETL server command line...

[Developer9]

@Chulett,

FTP stage is not working for this scenario as this mail box location (file and folder system is different).

Code: Select all

#!/usr/bin/bash
#------------------------------------------------------------------------------------
# Name        : FTP.sh							            
#---------------------------------------------------------------------------------------------------------------------------#
HOST= 'xxx.yyy.com'
user ='aaaa' 
password='bbbb'

ftp xxx.yyy.com
user $user $password
get 00123456789
quit
END_SCRIPT 
Exit 0

Below error message from the Director :

Code: Select all

FTP_ExecuteCommand..JobControl (@Sequencer_SH): Executed: /project_data/FTP/X/Data/FTP.sh
Reply=126
Output from command ====>
SH: /project_data/FTP/X/Data/FTP.sh: Permission denied

I did not specify any permissions for this file that might be the reason for this error .

While modifying the script and running jobs several iterations till the desired out put ..I welcome here the suggestions for the script modifications and Parameters to specify in execute command stage

Thank you

[chulett]

Yes... exactly. Get proper permissions on the shell script and (IMHO) get it working from the command line before you stick it in a DataStage job.

And brush up on the concept of here documents so your END_SCRIPT can actually be used. Example 19-6 in the linked site has an ftp example but there are tons more out there in the wild.

[chulett]

I'm also curious why your original script has a 'put' and your new script has a 'get'.

[Developer9]

@Chulett,

Reason for PUT :

Currently, customer is using windows script (mentioned in original) to place the file.The proposed process is to replace it with unix shell script executed by datastage process.

Reason for GET:(testing purpose only)

As part of my testing the ftp connection ,I am trying to read the file (already present) from target server(Production mail box ftp location ) make sure atleast there is a connection exist between unix server and Mail box location .

But the actual requirement is PUT/PLACE in the ftp mail box location

Thank you

[chulett]

You could have just said the get was for testing only. Everything else we already know.

ps. This isn't Twitter.

[Developer9]

@Chulett,

Agree with you ..too much information not necessary