Need to Untar the tar files

deepa.y · Post by **deepa.y** » Wed Feb 08, 2012 4:21 am

Hi,
I have Tar files which i need to Untar and read them without landing them in any directory.How can i read those files using parallel job?

pandeesh · Post by **pandeesh** » Wed Feb 08, 2012 4:30 am

Why you want to accomplish in a px job rather than unix command?
Check Compress and Expand stages, whether they satiate your requirement.

deepa.y · Post by **deepa.y** » Wed Feb 08, 2012 4:34 am

I checked Expand stage but it works only with Datasets.

kandyshandy · Post by **kandyshandy** » Wed Feb 08, 2012 4:39 am

Untar files in DS?

I guess you need to rely on some UNIX commands to achieve this.

I am not sure whether there is a new feature in 8.7!!

deepa.y · Post by **deepa.y** » Wed Feb 08, 2012 4:41 am

I came across some posts which had shown that it could be done using Named Pipe concept in server jobs.Is there any way to do that in parallel job?

ray.wurlod · Post by **ray.wurlod** » Wed Feb 08, 2012 6:33 am

What if you were to use tar as the filter command in a Sequential File stage?

deepa.y · Post by **deepa.y** » Wed Feb 08, 2012 7:13 am

Hi Ray,
Each tar file contains multiple folders and each folder inturn contain few files.Filter command is not working in this case as it cannot be read like file.

ray.wurlod · Post by **ray.wurlod** » Wed Feb 08, 2012 6:33 pm

You neglected to mention that little fact.

I don't believe there will be a solution that does not involve saving the content of the archive to disk.

qt_ky · Post by **qt_ky** » Fri Feb 10, 2012 2:15 pm

tar is a unix command and there is not a stage you can use out of the box that will read or write tar files without actually calling the tar command.

Could you share the links you found about using tar with named pipes?

anriliu · Post by **anriliu** » Thu Feb 16, 2012 8:47 pm

Why not considering embedded perl/python script in prerun step?

qt_ky · Post by **qt_ky** » Thu Feb 16, 2012 9:51 pm

Why not land the data? Is it to help performance or a lack of disk space?

Have you asked your UNIX admin about the named pipe method?

deepa.y · Post by **deepa.y** » Fri Feb 17, 2012 12:32 am

Hi,
As we have huge files,we cannot store it on disk.The issue is resolved.We are using External source stage to read the tar file to standard output and using them further .
tar -xOvf path of file.

qt_ky · Post by **qt_ky** » Fri Feb 17, 2012 7:08 am

It costs less to store huge files on disk than in memory. I'm just saying, that's what disk is for...

I don't see any tar -O option on my flavor of UNIX... What does -O do?

deepa.y · Post by **deepa.y** » Fri Feb 17, 2012 7:18 am

qt_ky wrote:I don't see any tar -O option on my flavor of UNIX... What does -O do?

-O is for standard output