Find a file name coming out of a folder stage

chulett · Post by **chulett** » Fri Aug 01, 2008 7:10 am

BIuser wrote:The problem is figuring out what happens when the stored procedure sends back either a "fail" or a "good to go" response. If the response is good to go then I need to parse the rows from the file. Will I be able to go to the stored proc and then process results in a Transformer and then revert back to looking at the folder stage again and take the processing from there ?

Me, I'd split this into two jobs: one that just takes the filename and calls the stored procedure and a second one that processes the file. Only run the second if the first one approves the file. And do all this in a Looping Sequence for each file, one at a time.

BIuser · Post by **BIuser** » Fri Aug 01, 2008 7:14 am

chulett wrote:

In your shoes (even though now I know they probably wouldn't be my style) I'd build a Looping Sequence to do this rather than fight with the Folder stage, especially if the files are of any size. The Folder stage has a well known problem with 'large' files and will tend to fall over dead if the contents of the file can't be stored in memory.

What 'large' means seems to vary from system to system and O/S to O/S. Some can handle hundreds of megabytes while we've seen others that can only handle tens. FYI.

Well thanks a mil for your input, I appreciate it ... ALOT. I am going to try this looping sequence and see what I can come up with. If not, you can rely on me being back here to nag in your ears !

So I won't mark this thread as resolved ... YET ... but i'll get there ...

BIuser · Post by **BIuser** » Fri Aug 08, 2008 4:14 am

BIuser wrote:
chulett wrote:

In your shoes (even though now I know they probably wouldn't be my style) I'd build a Looping Sequence to do this rather than fight with the Folder stage, especially if the files are of any size. The Folder stage has a well known problem with 'large' files and will tend to fall over dead if the contents of the file can't be stored in memory.

What 'large' means seems to vary from system to system and O/S to O/S. Some can handle hundreds of megabytes while we've seen others that can only handle tens. FYI.
Well thanks a mil for your input, I appreciate it ... ALOT. I am going to try this looping sequence and see what I can come up with. If not, you can rely on me being back here to nag in your ears !

So I won't mark this thread as resolved ... YET ... but i'll get there ...

I eventually changed my job to use a command stage. The folder stage turned out to be way too complicated for a simple task !