Folder Stage

Pierre · Post by **Pierre** » Mon Mar 10, 2008 5:30 am

Hi,

Seems DS Folder Stage tries to keep files it reads in memory. If yes, can someone explain me why ?

Thanks.

ArndW · Post by **ArndW** » Mon Mar 10, 2008 5:47 am

Hello and welcome to DSXchange, Pierre. IYou posted your question in the "Site/Forum" section, where it does not belong and it most likely won't get answered until it is in the correct forum.

It would help if you could also state why you think the files are held in memory - are you getting error messages?

Pierre · Post by **Pierre** » Mon Mar 10, 2008 6:01 am

ArndW wrote:Hello and welcome to DSXchange, Pierre. IYou posted your question in the "Site/Forum" section, where it does not belong and it most likely won't get answered until it is in the correct forum.

It would help if you could also state why you think the files are held in memory - are you getting error messages?

It's a response of Ray.Wurlod that makes me think that :
"Try processing fewer and/or smaller files, find some solution that does not require a Folder stage (which tries to keep the entire files in memory) - perhaps using a loop in a job sequence, or put mor ... "
The only message I have is :
"<Missing or NULL> (<Missing or NULL>): <Missing or NULL>" on 3 of 10 files (witch are between 176 and 204 Mo when the others are near 50 Mo)...

kcbland · Post by **kcbland** » Mon Mar 10, 2008 7:39 am

Folder stages read each file as a row of data. Therefore, a row is an entire file. The stage metadata uses only two columns: filename and file contents.

Pierre · Post by **Pierre** » Mon Mar 10, 2008 8:27 am

kcbland wrote:Folder stages read each file as a row of data. Therefore, a row is an entire file. The stage metadata uses only two columns: filename and file contents.

OK, I understand.
Thank you very much.

Pierre.