checking file existence, moving and writing to target table

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
thanush9sep
Premium Member
Premium Member
Posts: 54
Joined: Thu Oct 18, 2007 4:20 am
Location: Chennai

checking file existence, moving and writing to target table

Post by thanush9sep »

Hi all

The scenario that I am facing is,

1. I have to search for a file "SDK04BAM.txt" in a directory say, "/in2app/in2tst01/in2/TEST_FOLDER/"
2. if the file is present then I have to move the file to a another directory say, /in2app/in2tst01/in2/INPUT_FOLDER/
3. And at the same time I have to write "FILE EXISTS" in a table (JOURNAL)
4. If the file is not present in "/in2app/in2tst01/in2/TEST_FOLDER/" then I have to write "FILE DOES NOT EXIST" in the same table (JOURNAL) and abort the job.

code:
$IFNDEF JOBCONTROL.H
$INCLUDE DSINCLUDE JOBCONTROL.H
$ENDIF

File_handle = DSAttachJob("FILE_PRESENCE",DSJ.ERRWARN)
Equate SUN To "FILECHECKING"
DATEPART= trim(oconv(date()-1 , 'DDMY[2,2,4]'),'/','A')
FILENAME= "SDK04BAM.txt.":DATEPART
SRCFILE = '/in2app/in2tst01/in2/TEST_FOLDER/':FILNAME
TGTFILE = '/in2app/in2tst01/in2/INPUT_FOLDER/'

CHECKCMD = 'test -e':SRCFILE
CALL DSExecute(unix,CHECKCMD,outmsg,retcode)
IF retcode = 0 THEN
Call DSLogInfo("files exists",SUN)
PARAMERR = DSSetParam(File_handle,FILESTATUS,"FILE EXISTS")
COPYCMD = 'mv ':SRCFILE TGTFILE
END
IF retcode =1 THEN
Call DSLogInfo('files does not exist ',SUN)
PARAMERR = DSSetParam(File_handle,FILESTATUS,"FILE NOT EXISTS")
END

Note:
No error in compilation but when I run the job it did not stop (running).

This is my first code, I know I am going wrong in a big way.
Could someone instruct me whether I am in the right path and give me few tips so that I can proceed further

Regards
Ajay
sbass1
Premium Member
Premium Member
Posts: 211
Joined: Wed Jan 28, 2009 9:00 pm
Location: Sydney, Australia

Post by sbass1 »

I would write a script that would check the existence of the file and move it to the new directory. Either write "File Exists" / "File Does Not Exist" to STDOUT or pass a return code. Call this script from a ExecCmd stage in a sequence job. Pass the cmd output as a job parameter to your job which writes to a table.

You could also call the script as a filter in a sequential file stage. Use /dev/null as the input file.

HTH...
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

There's obviously code missing as there's nothing here to actually run a job. Does "FILE_PRESENCE" actually start and is what "did not stop" or did that happen before that? Use DSLogInfo() in your code to leave breadcrumbs along the way so you know where things go sideways.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

I think you need a space after "test -e". I am not sure the test command will work outside of ksh. You may need some other command to test to see if file exists or add ksh in front of test. I would print the output to the log see what it is doing.
Mamu Kim
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I've done similar things, so it should work - provided the space is there as you noted.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Since you're already in DataStage BASIC, you could simply use OpenSeq to test for existence. It takes its THEN clause if the file exists, and its ELSE clause if the file does not exist. Don't forget to CloseSeq before exiting.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply