Page 1 of 1

Generate Multiple Files without outputting key column value

Posted: Tue Dec 10, 2013 6:31 am
by JeroenDmt
I am creating multiple files through one sequential file stage using the "Generate Multiple Files" option.
The filename is based on a root file string plus the value of a key column, so that for each distinct value of the key column a new file is generated containing the key value in the file name.

For the output properties I have set:
- Write Method = Generate Multiple Files
- Exclude Partition String = True
- Key = <fieldname>
- Use Value in Filename=True
- Root File String = <base file name>

That works perfectly fine. However it forces me to include the key column in the output, and I do not want to include that in the output. I only want to use it to set the filenames of the output file.
Is there any way I can achieve that? (using the generate multiple files property)

Posted: Tue Dec 10, 2013 8:25 am
by chulett
I'll be curious if anyone can help as you're kind of out there on the bleeding edge of the product, playing with new functionality. You may need to involve your official support provider and then come back and tell us the answer. :wink:

Posted: Tue Dec 10, 2013 8:56 am
by JeroenDmt
Going the official way as well, but I am betting on two horses. Maybe someone here has ran into this problem before. There are always more people playing with new functionality.

If the official support provider horse wins, I will post the results here as well obviously ;)

Posted: Wed Mar 19, 2014 2:03 pm
by gsbrown
Was this ever resolved? I'm now running into the same issue. I need "File Name" as one of the output columns to use in the file name generation, but I don't want it as a column in my output file.

Reply

Posted: Wed Mar 19, 2014 9:38 pm
by ssnegi
make the unwanted key column the first field in the column definition. Then Use the Filter Property of sequential file.
put unix script : cut -d, -f2,n
n--> total number of columns
This will print all the columns except the first.

Posted: Thu Mar 20, 2014 1:33 am
by JeroenDmt
For now you have to use a workaround like a unix script like the one just mentioned.

The enhancement of the DataStage functionality is planned for the next major release according to IBM Product Management.

resolved

Posted: Mon Dec 07, 2015 10:24 am
by babbu9
I was able to generate multiple files using the "generate multiple files" option.
You need to define the Key field (Ex: Region) on which you would like to split the data. I also included the Root File string to point to the Directory where the files will be created. You specify the file name in the root file string

Ex: /....../InformatServer/Projects/Project1/TgtFiles/Region

and it created multiple files with prefix Region.part00004.001,Region.part00009.001,.....

I still am playing with the providing the filename in a format that I need.
An option would be to use shell script for changing filenames after job has finished.
But the data in each file is specific to the Region field in my source data and it seems to be working.