Question reg. FTP Stage

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Question reg. FTP Stage

Post by vivekgadwal »

As part of a project, I am trying to FTP two extract files, that I am generating, to Mainframe. Both the files are fixed width (100 characters wide, including filler) and I have two separate jobs for them as they are generated in seperate streams. Both the jobs have the same design:

Code: Select all

Sequential file stage --> FTP stage
The jobs are running fine and GDG's are getting created on Mainframe. But the problem is, when I check in the Mainframe, the file is chopped to 80 characters.
We have encountered a similar issue before and we ran a bat script with the "ftp command"

Code: Select all

quote list lrecl=<number>
as an after job subroutine and the issue got resolved then.

Coming to the problem now, I am trying to use the same command in FTP stage and I couldn't find the right place in the stage to run the ftp command. I tried in every available command property (telnet before cmd, after cmd, transaction cmd etc.) in vain. When used in the before mentioned properties, the job is aborting saying "cannot execute the telnet command" (that is because it is an ftp command). We also put the number of prompts for the telnet as '3' and tried to execute this command in one of the prompts. The jobs are running, but, again, the file is getting chopped off to 80 characters!

Can you please suggest me how to approach this? Any help is much appreciated!

Thanks.
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You don't use the FTP stage to issue an ftp command, it is all metadata driven and streaming. If you want to execute the command, script it and run the script after job as you have done.

In the job I would read the file as one long fixed width field of 100 characters and run it through the FTP stage in the same manner. Make sure to tell it to not add delimiters to the output if you do that.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

80 characters is a common limit, dating back to the days of punched cards. You have to find out what FTP option (it could be a "before" command) you need to use (on your system) to override this, so that your GTG has the desired line length.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Post by vivekgadwal »

ray.wurlod wrote:You have to find out what FTP option (it could be a "before" command) you need to use (on your system) to override this, so that your GTG has the desired line length.
Ray,

Thanks for your reply. Can you please explain further what you meant by the above sentence? As far as I know, we need to run that quote site lrecl=<num> ftp command in order to overwrite the 80 character limit and this brings me back to the issue again...where/how do I run that command?
As Craig pointed out, I am not able to run that in FTP stage. Running this as an after-job sub-routine in the previous job that generates the extract file is not possible as I am already utilizing that to do some other action.
chulett: If you want to execute the command, script it and run the script after job as you have done
If there are some errors during FTP, an FTP stage aborts and logs an error in the Director. With the after-job subroutine, we lose that functionality isn't it?!

Thanks for the help in advance!!
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You only lose functionality in an 'after job' subroutine if you don't code it in. So whatever you need it to log, code that in. If you need to abort the job, either log a fatal error or set the exit status to a non-zero value.

If you need the routine to do multiple things, no reason you can't code multiple steps into the routine if you like.

Haven't dealt with mainframes and wouldn't know a GTG if it bit me in the butt, but I don't think you can issue the quote site lrecl=<num> command by itself in the 'Telnet Before Command'. From what I recall, that is a separate telnet session that connects and disconnects before the ftp is attempted so I don't believe it would persist into the ftp session. Could be wrong, however. :?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's what I intended - the "telnet before command". Of course your mainframe security probably doesn't allow telnet so you may need to set up a more secure (ssh) approach. But it has to be in the same session.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Post by vivekgadwal »

Thanks Ray and Craig for your replies. As you both pointed out, I was not able to use the FTP command within the FTP stage. So, I just wrote an FTP script and executed the "quote site lrecl=<num>" command. I am running this script from DOS command line as a before job routine using ExecDOS. This is working for me.
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
Aru
Participant
Posts: 2
Joined: Wed Sep 16, 2009 12:46 pm

Question reg. FTP Stage

Post by Aru »

vivekgadwal wrote:Thanks Ray and Craig for your replies. As you both pointed out, I was not able to use the FTP command within the FTP stage. So, I just wrote an FTP script and executed the "quote site lrecl=<num>" command. I am running this script from DOS command line as a before job routine using ExecDOS. This is working for me.
Hi,

I have similar problem with the FTP Stage, while trying to transfer files between windows and MF.

Tried in vain for Telnet Before Command and Transaction Begin Commands. While searching through the DSX forum, came across your post.

Will you be able to share the contents of the ftp script used in the post "Question reg. FTP Stage "
viewtopic.php?t=121500&highlight=ftp+stage

Thanks for your help.

Regards
Aru Nangunoori
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Re: Question reg. FTP Stage

Post by vivekgadwal »

Aru wrote: I have similar problem with the FTP Stage, while trying to transfer files between windows and MF.

Tried in vain for Telnet Before Command and Transaction Begin Commands. While searching through the DSX forum, came across your post.

Will you be able to share the contents of the ftp script used in the post "Question reg. FTP Stage "
viewtopic.php?t=121500&highlight=ftp+stage

Thanks for your help.

Regards
Aru Nangunoori
Aru,

My advice on this is, if you are trying to execute any FTP commands (not telnet, but FTP commands like "quotesite") you should use it from Shell/DOS.

I do not have the script with me, as it was with a different client at that time (it has been 2 years almost, so I might not remember certain aspects of the script). All the script does is to open an FTP connection and execute the FTP commands. You see, if the FTP commands like "quotesite" work well when executed from command line. FTP stage does not have the capability to perform such things.

Hope this helps...
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
Aru
Participant
Posts: 2
Joined: Wed Sep 16, 2009 12:46 pm

Re: Question reg. FTP Stage

Post by Aru »

Aru,

My advice on this is, if you are trying to execute any FTP commands (not telnet, but FTP commands like "quotesite") you should use it from Shell/DOS.

I do not have the script with me, as it was with a different client at that time (it has been 2 years almost, so I might not remember certain aspects of the script). All the script does is to open an FTP connection and execute the FTP commands. You see, if the FTP commands like "quotesite" work well when executed from command line. FTP stage does not have the capability to perform such things.

Hope this helps...[/quote]

Vivek,
Thanks for quick response.
I am able to get a working FTP script at the command DOS prompt. Will try it as a end of the job subroutine to transfer to MF
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Re: Question reg. FTP Stage

Post by vivekgadwal »

Aru wrote:
Vivek,
Thanks for quick response.
I am able to get a working FTP script at the command DOS prompt. Will try it as a end of the job subroutine to transfer to MF
You're welcome Aru. Glad I could help! :D

Also, the one suggestion that I have with this is, use an Execute Command activity in a Sequence Job so that you will have more control. Even after-job sub-routine is fine too, but I am just saying. It depends on your scenario though.
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
nirav.parikh
Participant
Posts: 14
Joined: Thu Dec 13, 2007 2:57 am

Re: Question reg. FTP Stage

Post by nirav.parikh »

There is an FTP PrePUT command option available in the FTP Stage Properties.

You can use that to issue a site command. This site command will be executed before the file is put on the MF Server.

Let me know if this helps.

Regards,
Nirav
Thanks & Regards
Nirav
vivekgadwal
Premium Member
Premium Member
Posts: 457
Joined: Tue Sep 25, 2007 4:05 pm

Re: Question reg. FTP Stage

Post by vivekgadwal »

nirav.parikh wrote:There is an FTP PrePUT command option available in the FTP Stage Properties.

You can use that to issue a site command. This site command will be executed before the file is put on the MF Server.

Let me know if this helps.

Regards,
Nirav
Wow! IBM incorporated this functionality now? What version are you using?
Vivek Gadwal

Experience is what you get when you didn't get what you wanted
Post Reply