Reading multiple files in server jobs
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 110
- Joined: Mon Jan 11, 2010 4:22 am
Reading multiple files in server jobs
Hi
We had requirement to read from 6000 text files(.txt) and load them into single Oracle table.
We dont take any consideration of the format of the file(directly dump into oracle table).The output oracle table has only one column.
for example :
Input :
The test1.txt has 2 rows.
01*WOLAS*350-1*OBC-S*0%%N%12791%14387%726%570%698%636%473%719%607%855%732%841%739%632%853
01*ACCHK*700-1*OBC-A*0%%N%12793%14387%80%74%84%76%70%72%68%79%63%88%71%101%80
The test2.txt has 2 rows.
01*ACACRA*400-1*OBC-A*0%%N%12805%14367%19%16%16%17%18%17%21%18%17%18%22%20%15
01*OEINV*101-3*OBC-I*0%%N%12793%14387%142%124%150%154%118%206%246%251%186%186%154%158%178
Output :
The output oracle table will have 4 rows and one column.The entire value in the first row in the source file is written into first column of the table..
We are using 7.5,Server jobs in windows environment.Is there any optimized way we can do it, other than using multiple instance job and calling it multiple times which is very tedious?
Is there any stage which does this in Server jobs?
Thanks
Kiran
We had requirement to read from 6000 text files(.txt) and load them into single Oracle table.
We dont take any consideration of the format of the file(directly dump into oracle table).The output oracle table has only one column.
for example :
Input :
The test1.txt has 2 rows.
01*WOLAS*350-1*OBC-S*0%%N%12791%14387%726%570%698%636%473%719%607%855%732%841%739%632%853
01*ACCHK*700-1*OBC-A*0%%N%12793%14387%80%74%84%76%70%72%68%79%63%88%71%101%80
The test2.txt has 2 rows.
01*ACACRA*400-1*OBC-A*0%%N%12805%14367%19%16%16%17%18%17%21%18%17%18%22%20%15
01*OEINV*101-3*OBC-I*0%%N%12793%14387%142%124%150%154%118%206%246%251%186%186%154%158%178
Output :
The output oracle table will have 4 rows and one column.The entire value in the first row in the source file is written into first column of the table..
We are using 7.5,Server jobs in windows environment.Is there any optimized way we can do it, other than using multiple instance job and calling it multiple times which is very tedious?
Is there any stage which does this in Server jobs?
Thanks
Kiran
You need to treat them as one large file, either buy concatenating them all together first or by using the Filter option in the Sequential File stage and using 'cat/type *' to send the contents to std out, which the stage will then consume.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 110
- Joined: Mon Jan 11, 2010 4:22 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 15
- Joined: Mon Jul 27, 2009 8:43 pm
Need help
Hi Ray,ray.wurlod wrote:TYPE is the DOS equivalent of the UNIX cat command. That's why Craig mentioned both. ...
I am new to the Datastage. Can u please explain how to use TYPE in-order to
combine all the files data into single file.
srini
Re: Need help
You can use a copy command.
copy/b file1+file2 targetfile
copy/b file1+file2 targetfile
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Code: Select all
TYPE file1 file2 file3 ...
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: