UnZipping of files using Datastage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 7
- Joined: Tue Jun 20, 2006 3:53 am
UnZipping of files using Datastage
I have a requirement where in need of unzip the files using Unix/through datastage execute command.
Please let me know is this possible and give me the syntax for that and properties related to this execute command activity.
Kalyan
Please let me know is this possible and give me the syntax for that and properties related to this execute command activity.
Kalyan
Kalyanachakravarthy VS
-
- Premium Member
- Posts: 783
- Joined: Mon Jan 16, 2006 10:17 pm
- Location: Sydney, Australia
Do you mean the Execute Command stage in a Sequence job? If so, you use the exact same syntax there that you would from the command line. Only wrinkle that comes in is if you need to use Job Parameters in the call, any such need to be in the Parameters prompt for them to be 'expanded' properly.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
read a specific file from .tar.gz and transform it on fly
we need to parse and transform a .tar.gz file in memory. So I used a external source stage and tried to use the following command to tar, gzip and read a specific file inside the tar file.
tar -xzf xyz.tar.gz xx.tsv
Its not importing any records form it. The format is fine as I unzipped manually and then tried to import it. It works. So i used the same format with this command and it should wokr. But it isnt. Am I missing something??? Any suggestions please???? I need a solution asap. Thanks in advance.
More detail:
Need to untar (tar -xzf <tar_file> <specific file in tar>) only a specific file and then read & transform on the fly .
1: Using seq file to read.
2 : Using filter option in seq file stge and using the cmd (tar -xzf xyz.tar.gz xx.tsv)
3: not sure what to give in actual filename property
When i execute the job its not aborting but saying 0 rows imported/rejected.
tar -xzf xyz.tar.gz xx.tsv
Its not importing any records form it. The format is fine as I unzipped manually and then tried to import it. It works. So i used the same format with this command and it should wokr. But it isnt. Am I missing something??? Any suggestions please???? I need a solution asap. Thanks in advance.
More detail:
Need to untar (tar -xzf <tar_file> <specific file in tar>) only a specific file and then read & transform on the fly .
1: Using seq file to read.
2 : Using filter option in seq file stge and using the cmd (tar -xzf xyz.tar.gz xx.tsv)
3: not sure what to give in actual filename property
When i execute the job its not aborting but saying 0 rows imported/rejected.
battaliou wrote:If you unzip directly into a named pipe, you can read it directly as a sequential file without landing the data.
Ok, you need to direct the output of you tar to a named pipe. So create a named pipe called "np" by doing the following:
mkfifo np
When you're ready to run the job, simply go:
tar -xzf xyz.tar.gz xx.tsv > np
In datastage, your sequential file is called np and the data should stream through. You will need to run your job shortly after issuing the tar command, else the named pipe will time out.
mkfifo np
When you're ready to run the job, simply go:
tar -xzf xyz.tar.gz xx.tsv > np
In datastage, your sequential file is called np and the data should stream through. You will need to run your job shortly after issuing the tar command, else the named pipe will time out.
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.