Hi,
Is there any way in datastage i can split the input file (eg 1 GB) based on size lets say 1 MB and create the multiple files.
split data based on size in datastage
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 536
- Joined: Thu Oct 11, 2007 1:48 am
- Location: Bangalore
-
- Premium Member
- Posts: 536
- Joined: Thu Oct 11, 2007 1:48 am
- Location: Bangalore
Hi,
This is what i exactly suggested,but my client is rigid,want to check the capebility of datastage![Crying or Very sad :cry:](./images/smilies/icon_cry.gif)
This is what i exactly suggested,but my client is rigid,want to check the capebility of datastage
![Crying or Very sad :cry:](./images/smilies/icon_cry.gif)
Thanks
Prasoon
ETL Consultant
LinkedIn :- http://www.linkedin.com/profile/view?id ... ab_pro_top
Blog:- http://dsshar.blogspot.com/
Prasoon
ETL Consultant
LinkedIn :- http://www.linkedin.com/profile/view?id ... ab_pro_top
Blog:- http://dsshar.blogspot.com/
I'm not aware of any "split by size" option. Of course you could BuildOp whatever you like and still call it "in DataStage". Otherwise you would have to do something by row count after you figure out approximately how many records will generally equal 1MB. Using the Folder stage as a target is one option but is a bit of a pain in the butt. There have been other suggestions made in the past when this question was asked before that a search should turn up.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Since the input file is, by definition, a source it is not changed in the job. If you wish to use a UNIX utility such as "split" to split the file then you can do so as part of the before-job calls, likewise you could call a BASIC Before-job routine to perform more complex tasks on the file.
Are you sure you didn't mean to split the output file?
Are you sure you didn't mean to split the output file?
The Big Data File stage in 9.1 has an optional Max File Size property for writing target files. When the max size is reached (in MB), it generates another target file. I found this in the documentation; have not tested it myself.
Choose a job you love, and you will never have to work a day in your life. - Confucius