generate filename dynamically and pass as parameter

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
soham296
Participant
Posts: 33
Joined: Fri Jan 13, 2006 5:13 am

generate filename dynamically and pass as parameter

Post by soham296 »

Hi, How can I watch for a file in a particular directory whose name changes everyday?

We will have data from sequential file as daily feed and the filename will be like <some name>appended to today's date (for eg. seqfile_25.01.06.txt). This date suffix will change everyday. So, we'll need to build the string dynamically and pass it on the file watcher. In clues on how can i build a paramter dynamically and pass it on to a stage, say the file watcher stage.

Any other simpler ideas of achieving the same ??
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The PX sequential file stage lets you use a wildcard pattern for reading multiple files; this should be your simplest solution.
battaliou
Participant
Posts: 155
Joined: Mon Feb 24, 2003 7:28 am
Location: London
Contact:

Post by battaliou »

Assuming you clear the directory after use, I'd use a bit of BASIC here, combined with UNIX so you'd get something like:

CMD = 'ls /incoming/data'

LOOP
CALL DSExecute('UNIX',CMD,o1,o2)
UNTIL o1 <> '' DO
SLEEP 60
REPEAT

* Setup unix_cat, run it, wait for it to finish, and test for success
hJob1 = DSAttachJob("unix_cat", DSJ.ERRFATAL)
If NOT(hJob1) Then
Call DSLogFatal("Job Attach Failed: unix_cat", "JobControl")
Abort
End
ErrCode = DSSetParam(hJob1, "filename", o1<1>)
ErrCode = DSSetDisableProjectHandler(hJob1, @FALSE)
ErrCode = DSSetDisableJobHandler(hJob1, @FALSE)
ErrCode = DSRunJob(hJob1, DSJ.RUNNORMAL)
ErrCode = DSWaitForJob(hJob1)
Status = DSGetJobInfo(hJob1, DSJ.JOBSTATUS)
If Status = DSJS.RUNFAILED Or Status = DSJS.CRASHED Then
* Fatal Error - No Return
Call DSLogFatal("Job Failed: unix_cat", "JobControl")
End
3NF: Every non-key attribute must provide a fact about the key, the whole key, and nothing but the key. So help me Codd.
Post Reply