Source as a parameter?

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 12:31 am

Hi,

We are going to have a simple job which loads from a sequential file to a Oracle table.

But the problem here is since the volume of the records is more the data is splitted to n number of chunks of sequential file.

Basically we are trying to develop a single job and parameterizing the source sequential file name as a parameter.

Is this a good option.. if you guys have anyother thoughts let me know.

Thanks
maddi

ArndW · Post by **ArndW** » Wed May 25, 2005 12:35 am

Madhav,

this is a very practicable and intelligent approach, it makes sense to use a parameter for any value that can or will change in the future, and by using the path to a source file as a parameter you ensure this. Also, if you make a multi-instance job and your target is a database you can run this job several times in parallel.

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 3:34 am

Hey,
I used the following script...

PROJECT_NAME=beacon_dev
JOB_NAME=TRY_STGTgbCGRndBItem
DSJOB=$DSHOME/bin/dsjob

$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

$DSJOB -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

$DSJOB -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

1st sequential file data loaded successfully whereas for 2nd and 3rd run
it throws DSJ.BADSTATE error..

can anyone suggest me why this is occuring.

Thx
Maddi..

ArndW · Post by **ArndW** » Wed May 25, 2005 3:37 am

Check your log file in the director for the 1st run; it might have started but it ended up with an "Aborted" status. This is why the subsequent runs did not work.

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 3:46 am

In the log it says completed successfully.

FYI when I tried running the each DSJOB command one after another in the command line.. It is executing successfully!!

ArndW · Post by **ArndW** » Wed May 25, 2005 4:03 am

Madhav_M,

then you have a timing issue, the dsjob -run command is returning while the status of the job isn't quite finished. Before looking into an elegant solution, just put a "sleep 10" command between the dsjob -run call and see what happens.

ArndW · Post by **ArndW** » Wed May 25, 2005 4:04 am

Sorry, I just took a quick look at the dsjob options and you need to specify the "-wait" in order to wait for a completed status on subsequent runs.

dsxdev · Post by **dsxdev** » Wed May 25, 2005 5:27 am

This is absolutely right you have to uses

Code: Select all

dsjob -wait

option to run the jobs.

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 7:22 am

Excellent it works fine..

Thanks
Maddi

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 7:29 am

hey I tried sleep only..-wait option seems to be not working for me..

I tried like this,

$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME

ArndW · Post by **ArndW** » Wed May 25, 2005 7:40 am

Madhav_M,

the "-wait" option tells the dsjob call not to return until the job has finished; So you need it on your first call as well in subsequent ones.

killbill1 · Post by **killbill1** » Wed May 25, 2005 10:47 am

Madhav,

Why don't you enable "Allow Multiple Instance" option from the respective job properties and run the job with different extension each time, that way you don't have to use sleep or wait.

Example:

$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst1

$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst2

$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst3

Madhav_M · Post by **Madhav_M** » Wed May 25, 2005 10:03 pm

Hi,
Killbill1 i'll try ur option too.

Anrdw: dsjob - wait option throws me error like invalid arugument!
I don't find -wait option at all?

ArndW · Post by **ArndW** » Thu May 26, 2005 1:46 am

Madhav_M,

try just entering "dsjob -run" on your command line, you will get a short help list. The "-wait" option exists and is heavily used, so it will work; you might be making a simple syntax error when calling it.

ray.wurlod · Post by **ray.wurlod** » Thu May 26, 2005 6:59 am

Just as a point of interest, if the text file is fixed-width format, then a parallel job can automatically read it in parallel.
Because the row size and file size are known, the beginning and ending offset for each read can be pre-calculated. Therefore multiple read processes can be applied to the import operator.