FTP Enterprise stage

[sam334]

Hi all,
We have a job which reads a .txt file through FTP enterprise stage and bulk load the data to DB2. Issue is , the job is copying the file which it reads through FTP stage and the file is stored in parallel resource folder in DS server.

Job design-

FTP Stage-- Transformer---Bulk Load to DB2
Any thought on this. has anyone come across this issue before.

Thanks.

[qt_ky]

Personally I think it's a better practice to land 100% of such data locally in a first job before calling a separate job that will read the local file and load it into a database.

[FranklinE]

I've not seen this at all, over almost three years using FTP Enterprise as the first stage in multi-stage jobs.

I don't agree that landing the files first is preferable even most of the time. It's first a redundant use of disk on the local server, second doesn't require clean-up of files afterwards (except in Sam's case, which I don't understand how it works).

But dueling opinions aside, the decision point is resources. Do you have a luxury of excess disk? Land the files, by all means. Is your FTP channel restricted (mine is)? Timing becomes critical, and again landing files can be a better option. But for a cycle with dozens of downloads representing close to terabytes in volume, landing files may just be the worse idea.

[sam334]

Ok. Thanks. I even have no heard about the issue. So far we have a windows script to delete the file from that folder. Lets see.