sequential file pattern property

[bhargav_dd]

Hi
I have a requirement where I have to fetch few files from a folder of similar pattern and having same metadata and then need to apply some transformations on it and then I have to produce same no of files fetched from the source folder and place them in target folder.

Ex:
dir1 has 210 files and i need to fetch some 50 files of similar pattern ( which I specified in my seq file file pattern stage property)
after applying transformation it need to produce 50 files in target folder

I have problem in splitting individual files which are being read from input folder after the transformation is done (no of files in i/p folder are not fixed and no of records in each i/p file are also not fixed )
can anyone post me a solution for this

[jwiles]

Two possible options:

1) Write a job sequence which builds a list of the input files then runs a loop which calls the job once for each file.

2) Use an External Target stage to call an awk script or other custom logic to split the output data as it's being written. Create a column indicating the target output file for each row.

Sequential File stage doesn't support writing to multiple output files.

Regards,

[chulett]

Another option - writing to a Folder stage in a Server job.

[madhukar]

concat all files and filename field into one file.
read file, transform and output
do the inverse of concat to reproduce files

:D

[ray.wurlod]

And what, pray tell, is the "inverse of concat"? (Hint: it's not "tacnoc".)

[chulett]

Oooo, oooo, I know! It's... wait, what? It's not?