DSXchange

Hi,
I am having a big DB2 table in terms of number of rows. After passing through various stages I need to write into a flat file, so I am writing into a .csv file.
Now as the number of records are very large in terms of billions, I want to store the result into multiple files with same name with last 2 or 3 characters as suffixes like.

flatfile_000.csv
flatfile_001.csv
flatfile_002.csv
.
.
.
.
Like this I want to create my target file. I am supposed to make each file with 50,000 records each.

Can anybody tell me how can I achieve this?

Thanks
Kavuri

with a billion records and 50,000 records per file, u will need 20,000 files.
Why not store in DataSet? It will take much less space. And easy for further processing as well.

Or perhaps explain why you 'need' to write to a flat file.

Hi,
Target flat files are utilised by another product which is written in Orchestrate. So i am writing them in .csv format. If you have any other idea please let me know. Or please tell me how can I achieve this? If not 20,000 we will prepare more flat files. That is what the requirement from the team.

Thanks
Kavuri

Create one large file using DataStage then use the UNIX command split to break it into many smaller files.

kavuri wrote:...Target flat files are utilised by another product which is written in Orchestrate...

Are you sure? Orchestrate is the old name of the PX/EE product so you might as well use DataSets.