Size limit for Folder Stage
Moderators: chulett, rschirm, roy
Size limit for Folder Stage
All,
In my job in the source we have a Folder stage then some transformers to velidate Source file data and then target folder stage.My job is not able to read file of size more than 65 MB.Is there any limitation of the Folder stage with maximum file size it can read or Write.
In my job when ever folder stage encounters a file with size > 65MB it aborts.
Is there any means by which I can over come this limitaion.
Please let me know your suggestions.
Thanks in Advance.
Amit
In my job in the source we have a Folder stage then some transformers to velidate Source file data and then target folder stage.My job is not able to read file of size more than 65 MB.Is there any limitation of the Folder stage with maximum file size it can read or Write.
In my job when ever folder stage encounters a file with size > 65MB it aborts.
Is there any means by which I can over come this limitaion.
Please let me know your suggestions.
Thanks in Advance.
Amit
Hi I'm a data stage developer.
Re: Size limit for Folder Stage
Dunno of such limitations, except when enforced at the OS level. Ask your UNIX admin about the size constraints.
gateleys
gateleys
Yes, there are known but undocumented limits on the sizes of files that can be sucked thru a Folder stage. It seems to vary from version to version and O/S to O/S. When you find those limits, as far as I know the only thing you can do about that is ensure that your files are all under that limit - even if it means some kind of preprocessing is in order.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 12
- Joined: Wed Apr 19, 2006 7:41 am
Re: Size limit for Folder Stage
Hi AmitAmit Nema wrote:All,
In my job in the source we have a Folder stage then some transformers to velidate Source file data and then target folder stage.My job is not able to read file of size more than 65 MB.Is there any limitation of the Folder stage with maximum file size it can read or Write.
In my job when ever folder stage encounters a file with size > 65MB it aborts.
Is there any means by which I can over come this limitaion.
Please let me know your suggestions.
Thanks in Advance.
Amit
I dont think the size of folder will matter in any ways as far as DataStage is concerned.Because,I have tested the folder stage with 90 MB size folder and I met with success.May be problem with O/S.
Regards
Surendra
Re: Size limit for Folder Stage
As I noted, is does matter.surendra_ds wrote:I dont think the size of folder will matter in any ways as far as DataStage is concerned.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Size limit for Folder Stage
Thanks everyone for your suggestions.We are using Unix at the server end and win XP at the client end.Might be the limitation of Unix.
But through Unix when i try to Copy or move a file with more that 90 MB it works fine.
If any UNIX expert can through some light on this would be of great help.
Thanks,
Amit
But through Unix when i try to Copy or move a file with more that 90 MB it works fine.
If any UNIX expert can through some light on this would be of great help.
Thanks,
Amit
Hi I'm a data stage developer.
Ok... one more time. This is a DataStage limitation, not a UNIX limitation. It is a known issue with the Folder stage.
Open up a support case with Ascential, see if there is any work-around or perhaps a patch they can offer. I'm not aware of one, unfortunately, other than to process smaller files.
Open up a support case with Ascential, see if there is any work-around or perhaps a patch they can offer. I'm not aware of one, unfortunately, other than to process smaller files.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers