Source as a parameter?
Moderators: chulett, rschirm, roy
Source as a parameter?
Hi,
We are going to have a simple job which loads from a sequential file to a Oracle table.
But the problem here is since the volume of the records is more the data is splitted to n number of chunks of sequential file.
Basically we are trying to develop a single job and parameterizing the source sequential file name as a parameter.
Is this a good option.. if you guys have anyother thoughts let me know.
Thanks
maddi
We are going to have a simple job which loads from a sequential file to a Oracle table.
But the problem here is since the volume of the records is more the data is splitted to n number of chunks of sequential file.
Basically we are trying to develop a single job and parameterizing the source sequential file name as a parameter.
Is this a good option.. if you guys have anyother thoughts let me know.
Thanks
maddi
Madhav,
this is a very practicable and intelligent approach, it makes sense to use a parameter for any value that can or will change in the future, and by using the path to a source file as a parameter you ensure this. Also, if you make a multi-instance job and your target is a database you can run this job several times in parallel.
this is a very practicable and intelligent approach, it makes sense to use a parameter for any value that can or will change in the future, and by using the path to a source file as a parameter you ensure this. Also, if you make a multi-instance job and your target is a database you can run this job several times in parallel.
Hey,
I used the following script...
PROJECT_NAME=beacon_dev
JOB_NAME=TRY_STGTgbCGRndBItem
DSJOB=$DSHOME/bin/dsjob
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
1st sequential file data loaded successfully whereas for 2nd and 3rd run
it throws DSJ.BADSTATE error..
can anyone suggest me why this is occuring.
Thx
Maddi..![Rolling Eyes :roll:](./images/smilies/icon_rolleyes.gif)
I used the following script...
PROJECT_NAME=beacon_dev
JOB_NAME=TRY_STGTgbCGRndBItem
DSJOB=$DSHOME/bin/dsjob
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
1st sequential file data loaded successfully whereas for 2nd and 3rd run
it throws DSJ.BADSTATE error..
can anyone suggest me why this is occuring.
Thx
Maddi..
![Rolling Eyes :roll:](./images/smilies/icon_rolleyes.gif)
This is absolutely right you have to uses option to run the jobs.
Code: Select all
dsjob -wait
Happy DataStaging
hey I tried sleep only..-wait option seems to be not working for me..
I tried like this,
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
I tried like this,
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
Madhav,
Why don't you enable "Allow Multiple Instance" option from the respective job properties and run the job with different extension each time, that way you don't have to use sleep or wait.
Example:
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst1
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst2
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst3
Why don't you enable "Allow Multiple Instance" option from the respective job properties and run the job with different extension each time, that way you don't have to use sleep or wait.
Example:
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst1
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst2
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst3
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Just as a point of interest, if the text file is fixed-width format, then a parallel job can automatically read it in parallel.
Because the row size and file size are known, the beginning and ending offset for each read can be pre-calculated. Therefore multiple read processes can be applied to the import operator.
Because the row size and file size are known, the beginning and ending offset for each read can be pre-calculated. Therefore multiple read processes can be applied to the import operator.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.