Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.
Moderators: chulett , rschirm , roy
wahi80
Participant
Posts: 214 Joined: Thu Feb 07, 2008 4:37 pm
Post
by wahi80 » Thu Jan 24, 2013 11:14 am
Hi,
I have a job where I read multiple files using the File pattern approach.
I also needed to get the file name , so I used the filename column option.
My file pattern is ??.txt.
While my job read all the data in the files correctly, the name in the filename column is always ??.txt. I was expecting to get 1a.txt, 2b.txt and so on.
Is this the way it is designed to work?
prasson_ibm
Premium Member
Posts: 536 Joined: Thu Oct 11, 2007 1:48 am
Location: Bangalore
Post
by prasson_ibm » Thu Jan 24, 2013 11:26 am
In case of file pattern,datastage perform just like unix "cat" command,since all the metadata of files are same,so its read bunches of the files togather.
wahi80
Participant
Posts: 214 Joined: Thu Feb 07, 2008 4:37 pm
Post
by wahi80 » Thu Jan 24, 2013 11:45 am
But that is not my requirement, I need to map filename to each row
chetan.c
Participant
Posts: 112 Joined: Tue Jan 17, 2012 2:09 am
Location: Bangalore
Post
by chetan.c » Thu Jan 24, 2013 12:47 pm
Set this environment variable APT_IMPORT_PATTERN_USES_FILESET to true and check.
wahi80
Participant
Posts: 214 Joined: Thu Feb 07, 2008 4:37 pm
Post
by wahi80 » Thu Jan 24, 2013 12:50 pm
That works!!!
chulett
Charter Member
Posts: 43085 Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO
Post
by chulett » Thu Jan 24, 2013 1:56 pm
FYI - a search here would have turned that information up.
-craig
"You can never have too many knives" -- Logan Nine Fingers
chetan.c
Participant
Posts: 112 Joined: Tue Jan 17, 2012 2:09 am
Location: Bangalore
Post
by chetan.c » Fri Jan 25, 2013 12:39 pm
Apologies Craig.
I Should have pointed towards it.
chulett
Charter Member
Posts: 43085 Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO
Post
by chulett » Fri Jan 25, 2013 1:54 pm
No worries! Just wanted to add that little tidbit.
-craig
"You can never have too many knives" -- Logan Nine Fingers