Reading Bunch of Zip files using File pattern

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Hope
Participant
Posts: 97
Joined: Sun May 13, 2007 2:51 pm
Contact:

Reading Bunch of Zip files using File pattern

Post by Hope »

I have bunch of Zip files. All the file names start with same name ex:
file1
file2
file3
.
.
.
.
.
filen
I am using FilePattern. But all my files are Zip files.How can I read these Zip files as Filepattern does not have the filter option.
Please help.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Like putting your right elbow in your left ear some things are impossible. You need to pre-process the zipped files with a script that unzips them, then you can use a File Pattern in the Sequential File stage to process the unzipped files.
Or you could create a loop in a job sequence that unzips one at a time and executes a job to process that particular file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hope
Participant
Posts: 97
Joined: Sun May 13, 2007 2:51 pm
Contact:

Post by Hope »

I did that in before job subroutine but I have space issues. When I unzip the files I get the error " No disk space". I am reading all the files in the sequnetial files and using the filter=gunzip . If I use this will I have any space issues?.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The simplest solution is to put all the zip files in a directory by themselves so you don't need a filter. You could also use an external stage with a command like "uncompress -c *Z" to uncompress the files to the data stream (this version will keep the original compressed files on disk).
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Hope wrote:I did that in before job subroutine but I have space issues. When I unzip the files I get the error " No disk space".
Disk is cheap.
Get more disk.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply