Source as a parameter?
Moderators: chulett, rschirm, roy
Source as a parameter?
Hi,
We are going to have a simple job which loads from a sequential file to a Oracle table.
But the problem here is since the volume of the records is more the data is splitted to n number of chunks of sequential file.
Basically we are trying to develop a single job and parameterizing the source sequential file name as a parameter.
Is this a good option.. if you guys have anyother thoughts let me know.
Thanks
maddi
We are going to have a simple job which loads from a sequential file to a Oracle table.
But the problem here is since the volume of the records is more the data is splitted to n number of chunks of sequential file.
Basically we are trying to develop a single job and parameterizing the source sequential file name as a parameter.
Is this a good option.. if you guys have anyother thoughts let me know.
Thanks
maddi
Madhav,
this is a very practicable and intelligent approach, it makes sense to use a parameter for any value that can or will change in the future, and by using the path to a source file as a parameter you ensure this. Also, if you make a multi-instance job and your target is a database you can run this job several times in parallel.
this is a very practicable and intelligent approach, it makes sense to use a parameter for any value that can or will change in the future, and by using the path to a source file as a parameter you ensure this. Also, if you make a multi-instance job and your target is a database you can run this job several times in parallel.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Hey,
I used the following script...
PROJECT_NAME=beacon_dev
JOB_NAME=TRY_STGTgbCGRndBItem
DSJOB=$DSHOME/bin/dsjob
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
1st sequential file data loaded successfully whereas for 2nd and 3rd run
it throws DSJ.BADSTATE error..
can anyone suggest me why this is occuring.
Thx
Maddi..
I used the following script...
PROJECT_NAME=beacon_dev
JOB_NAME=TRY_STGTgbCGRndBItem
DSJOB=$DSHOME/bin/dsjob
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
1st sequential file data loaded successfully whereas for 2nd and 3rd run
it throws DSJ.BADSTATE error..
can anyone suggest me why this is occuring.
Thx
Maddi..
Check your log file in the director for the 1st run; it might have started but it ended up with an "Aborted" status. This is why the subsequent runs did not work.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Madhav_M,
then you have a timing issue, the dsjob -run command is returning while the status of the job isn't quite finished. Before looking into an elegant solution, just put a "sleep 10" command between the dsjob -run call and see what happens.
then you have a timing issue, the dsjob -run command is returning while the status of the job isn't quite finished. Before looking into an elegant solution, just put a "sleep 10" command between the dsjob -run call and see what happens.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Sorry, I just took a quick look at the dsjob options and you need to specify the "-wait" in order to wait for a completed status on subsequent runs.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
This is absolutely right you have to uses option to run the jobs.
Code: Select all
dsjob -wait
Happy DataStaging
hey I tried sleep only..-wait option seems to be not working for me..
I tried like this,
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
I tried like this,
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME
Madhav_M,
the "-wait" option tells the dsjob call not to return until the job has finished; So you need it on your first call as well in subsequent ones.
the "-wait" option tells the dsjob call not to return until the job has finished; So you need it on your first call as well in subsequent ones.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Madhav,
Why don't you enable "Allow Multiple Instance" option from the respective job properties and run the job with different extension each time, that way you don't have to use sleep or wait.
Example:
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst1
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst2
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst3
Why don't you enable "Allow Multiple Instance" option from the respective job properties and run the job with different extension each time, that way you don't have to use sleep or wait.
Example:
$DSJOB -run -param fname=CRGBNXTRCT06_01.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst1
$DSJOB -wait -run -param fname=CRGBNXTRCT06_02.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst2
$DSJOB -wait -run -param fname=CRGBNXTRCT06_03.txt -mode NORMAL $PROJECT_NAME $JOB_NAME.Inst3
Madhav_M,
try just entering "dsjob -run" on your command line, you will get a short help list. The "-wait" option exists and is heavily used, so it will work; you might be making a simple syntax error when calling it.
try just entering "dsjob -run" on your command line, you will get a short help list. The "-wait" option exists and is heavily used, so it will work; you might be making a simple syntax error when calling it.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Just as a point of interest, if the text file is fixed-width format, then a parallel job can automatically read it in parallel.
Because the row size and file size are known, the beginning and ending offset for each read can be pre-calculated. Therefore multiple read processes can be applied to the import operator.
Because the row size and file size are known, the beginning and ending offset for each read can be pre-calculated. Therefore multiple read processes can be applied to the import operator.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.