Reading multiple files with header
Moderators: chulett, rschirm, roy
Reading multiple files with header
Hi everyone ,
I am using a sequential file stage . and reading multiple files with Read Method set as "File Pattern" . The problem which i'm facing is .....all files are having header in it , although i'm specifying "First Line is Column Name" to TRUE ..... the header is also getting inserted into the target except the first file's header .
I do not want to edit the multiple source files (like removing the header from each of them) .
I do not find the solution in previous posts for this problem . If anybody knows... kindly provide the solution .
I am using a sequential file stage . and reading multiple files with Read Method set as "File Pattern" . The problem which i'm facing is .....all files are having header in it , although i'm specifying "First Line is Column Name" to TRUE ..... the header is also getting inserted into the target except the first file's header .
I do not want to edit the multiple source files (like removing the header from each of them) .
I do not find the solution in previous posts for this problem . If anybody knows... kindly provide the solution .
Sweta
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Ya , that could be done .. but all the columns in source are of varchar type so ..it smoothly getting transferred to the output .
Anyways .. i can handle that partcular header record in the transformer but i wanted to know is not there any option available in sequential file stage itself like the Filter thing ??
Anyways .. i can handle that partcular header record in the transformer but i wanted to know is not there any option available in sequential file stage itself like the Filter thing ??
Sweta
-
- Participant
- Posts: 4
- Joined: Mon Jul 05, 2010 1:21 am
You are reading multiple files from file pattern. Consider, if fields f1, f2, f3 and so on are there in files. Now go to filter stage and there set the condition as:
f1<>'f1'. This way only file records will be passed, not the column names.
E.g.
file1: f1, f2, f3
1, 1, 1
2, 2, 2
file2: f1, f2, f3
3,3,3
4,4,4
file3: f1, f2, f3
5,5,5
6,6,6
When you read the file pattern, records will come like as:
f1, f2, f3
1, 1, 1
2, 2, 2
f1, f2, f3
3,3,3
4,4,4
f1, f2, f3
5,5,5
6,6,6
Now, in filter stage if you define the constraint like this:
f1<>'f1'
Then from output link of filter you will get the result as:
1, 1, 1
2, 2, 2
3,3,3
4,4,4
5,5,5
6,6,6
.
Hope this solves your purpose
f1<>'f1'. This way only file records will be passed, not the column names.
E.g.
file1: f1, f2, f3
1, 1, 1
2, 2, 2
file2: f1, f2, f3
3,3,3
4,4,4
file3: f1, f2, f3
5,5,5
6,6,6
When you read the file pattern, records will come like as:
f1, f2, f3
1, 1, 1
2, 2, 2
f1, f2, f3
3,3,3
4,4,4
f1, f2, f3
5,5,5
6,6,6
Now, in filter stage if you define the constraint like this:
f1<>'f1'
Then from output link of filter you will get the result as:
1, 1, 1
2, 2, 2
3,3,3
4,4,4
5,5,5
6,6,6
.
Hope this solves your purpose
Varun Negi
Assistant System Enginner
TCS
Assistant System Enginner
TCS
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: