importing .gz file from unix server and unzip

kum_d · Post by **kum_d** » Wed Jul 18, 2007 9:56 pm

Hi

Can i know how to access the zip (.gz) files which are in remote unix server into windows client and unzip as well as rename to .txt file ,the same using DS

Please help me regrarding this?

DSguru2B · Post by **DSguru2B** » Wed Jul 18, 2007 10:02 pm

Unzip and ftp it over to the windows server. Write a shell that does all this.

ArndW · Post by **ArndW** » Wed Jul 18, 2007 10:20 pm

Are you using Server or PX to do this? You can ftp the files over first, then unzip them on Windows (this saves I/O overhead on the network); you could do this outside of DataStage in a script or within a job.

kum_d · Post by **kum_d** » Wed Jul 18, 2007 10:20 pm

Hi DSGuru,

The current process is they are accessing the .gz file which is in remote unxi server using FTP tool and moving to our local directory in windows.

But now we need to automate it using Datastage i.e we need to access and unzip after import using datastage.

I dont have idea about shell as you mentined me to use shell.

Can you plesae help me how can i do this using datastage

ray.wurlod · Post by **ray.wurlod** » Wed Jul 18, 2007 10:35 pm

Control the FTP separately from DataStage, or from an Execute Command activity in a job sequence, or from a before-job subroutine. Use a Filter command in a Sequential File stage to execute your unzip utility; the Sequential File stage will read stdout from the filter command if one is specified.

DSRajesh · Post by **DSRajesh** » Thu Jul 19, 2007 5:52 pm

Hi ArndW,

Can we import the .gz files which are in remote unix server using FTP satge in datastage or can we only access the sequential files using ftp stage?

I need to implement this using datastage only but not like using shell scripts or ftp tool out of datastage.

can you please help me here.

chulett · Post by **chulett** » Thu Jul 19, 2007 6:13 pm

No, the FTP stage cannot handle compressed or zip or binary (etc) files. It does a metadata driven record by record transfer rather than just streaming bytes, which means text files only.

"I need to implement this using datastage only"

I don't understand restrictions on using scripts or the O/S for some aspects of your work. No tool can do everything and it's perfectly OK to leverage shell scripts / batch files or external tools where appropriate. And this is one of those appropriate times.

Your DataStage job can run / monitor said shell script without issue.

DSRajesh · Post by **DSRajesh** » Thu Jul 19, 2007 6:26 pm

Hi chulett,

I dont have know about shell scripts.
Can you please help me out about the shell script code which works for this.

can you plesae give me an idea with sample shell script and how can we use it?

This helps me a lot to improve knowldge.

chulett · Post by **chulett** » Thu Jul 19, 2007 6:37 pm

Are you and "kum_d" working together on this? If not, do you have DataStage installed on a UNIX server? If so, there must be someone in your organization that can help you with a simple shell script like that.

ArndW · Post by **ArndW** » Thu Jul 19, 2007 6:53 pm

DSRajesh wrote:...Can we import the .gz files which are in remote unix server using FTP satge in datastage...

Sort of - but I wouldn't recommend doing it that way. If you "fake" the metadata in the FTP stage to be one (binary) column and pipe the incoming binary stream into a unzip program it will work (I have done that before) but that would normally not be a good approach. I think I used that solution in a case where the network connection was stable but very slow and it would have taken hours for the FTP to complete; by using the FTP stage in DataStage as mentioned above I was able to process the data as it came in instead of having to wait for all of it to arrive. Unless you have a similar scenario I would say that the answer to the ftp and gzipped file is "no"