Pick up new files only

[gomez]

Hi:
I have a requirement to pick up files from a landing directory. the landing directory keeps the file for 7 days at that location. now everytime i want to pick up a file i have to make sure I am picking up only files which have not been picked up already. Appreciate any suggestionson how to achieve this ?

Thanks

[asitagrawal]

May try to use DOS Command, DIR to list the files sorted by date modified and date created.
You have the list of files to chose

[ArndW]

Gomez - you will need to produce some code to do this logic. There are many different ways in which people have done this. If you can modify the files in that directory once you have picked them up (i.e. chmod to remove read access) you can use simple UNIX commands to select only those with appropriate access rights; this is a simple approach. Or you can store a list of files picked up in a database and compare the directory contents with the table.
I prefer to keeps things simple and copy the files elsewhere after they have been "picked up" so that there is no chance at ambiguity.

[ray.wurlod]

If you insist upon using DataStage you can write a server routine and invoke it from a job sequence. The STATUS statement can report the date and time a file was created, modified and accessed.

[sachin1]

hello, if i interpret your issue properly, i think in your directory, your getting a files which are used in datastage. just for example if you have a file named test.txt which is used in datastage, you are aware of the name of file that will be used in job, so what you can do is use an after job subroutine

ExecDos and pass a parameter to rename or move a file, as test_finished.txt.

ExecDos move #directory#\out\test.txt #directory#\out\test_finished.txt

[gomez]

Thanks for all your inputs. I understand there has to be some coding done to achieve this.
ArndW /Sachin- I wouldnt be able to move/rename the picked up files to another folder because there is a program which retrieves the files from vendor. If it doesnt find the file in the stage area, it will copy the file again which to DataStage would look like a new file.
I would try and maintain a Database table and then look up the table before picking up the file
Ray - I like the idea of using STATUS. I shall try that too and post my responses
Thanks a lot everyone for the help!

[ray.wurlod]

Note that it's the STATUS statement not the STATUS() function.