Create output files dynamically

deepa.y · Post by **deepa.y** » Fri Mar 30, 2012 12:52 am

Hi,
I need to create output files for distinct values in col1 and put corresponding data into those files.
input file:
col1,col2
1,xx
1,yy
2,zz
So,i read the posts and implemented the logic by using the awk command
awk '{nPosField1=index($0,",");print substr($0,nPosField1+1)>substr($0,1,nPosField1-1)}' in external target stage
by giving input as file name in column1 and data in column2 from a transformer.
This worked fine when my record delimeter is end or newline.
I need my record delimeter to be \x00 and final delimeter to be none.
so that my Output will be
out_1.txt->1,xx^@1,yy^@
out_2.txt->2,zz^@

In this case the above command will not hold good as the records are not in new line.Please suggest me how to achieve this?

kandyshandy · Post by **kandyshandy** » Fri Mar 30, 2012 1:46 am

I guess posting this in UNIX forum would help

qt_ky · Post by **qt_ky** » Fri Mar 30, 2012 5:46 am

How about using a Sequential File stage as the target instead of an awk command? Sequential File stage lets you choose delimiters and newline settings.

chulett · Post by **chulett** » Fri Mar 30, 2012 7:53 am

You might also look into the csplit function.

ray.wurlod · Post by **ray.wurlod** » Fri Mar 30, 2012 2:54 pm

Server jobs have a Folder stage that does this. You could use one of those in a server Shared Container in a parallel job, or maybe do all of your processing in a server job.