Running job parallel by passing invocation ID as parameter ?

[Developer9]

Hello Everyone;

Environment :Data Stage 7.5.2, AIX server , DB2 UDB

Requirement:To generate data sets corresponding to the different table names using parallel process..

Ex:Employee table has different load tables like Employee_01,Employee_02 (loaded weekly,monthly etc..)Have same "meta data"

Employee_01 corresponding file emp_01.ds
Employee_02 .......................emp_02.ds
Employee_03 ..................emp_03.ds


we have a job " DB2 table read and writes to a Dataset "

[

Code: Select all

DB2 API (Employee)>>>>Dataset(emp.ds)

]

Given "table name" and "file name" as parameter to the above job.
Job properties:Allow multiple instances job

SQL query :Select ENAME,SAL,JOB ...FROM #TABLE NAME#
File name:#emp.ds#

Here we can pass "Invocation ID as 1,2,3...manually "to generate these files".

Questions:

How can we automate these"Invocation ID"to generate files in multiple invocation job?

or

can we implement any server "job control routine" to call table names , file names and Invocation ID in "Batch job" to generate files?


Thank you.

[ray.wurlod]

1. Simply pass the invocation ID as a parameter (as well as appearing as part of the job name). If this job is being called from a multi-instance sequence, then the sequence's invocation ID can be obtained via a DS macro or Routine activity.

2. Yes. But are your DataStage BASIC programming skills up to it?

[qt_ky]

I did something similar on 8.5. This may help, or not...

One multi-instance job using RCP with TABLE as a parameter. In my case source was mainframe DB2 and target SQL Server with different metadata per table. I used one script calling dsjob and passed the table name as a parameter to the job as well as using the table name for the invocation ID. Same can be done from a a sequence job. Invocation ID does not have to be 1, 2, 3, ...; it can be text.

[Developer9]

@ray,

Thanks

1."multi-instance sequence"..(I am not familiar with this kind of approach )

2.Yes..I do have some Batch jobs runs using "job control routine"program

Ex:Master Batch :processA that triggers subsequent jobs by taking
Job Parameters :"parameter file "and "job dependency file (excel file)"

subsequent Batch:Process B1,B2,
Final Batch :Process F

Here I am looking to include "table name","file name" and "Invocation id "in the "job dependency file "to "run multiple instances"using this file . I will post the sample job control program if needed

Code: Select all

Job dependency file format:

Column 1:Jobname  (jobemp.1,jobemp.2,jobemp.3 so on)

column 2:Table name (emp_01,emp_02,emp_03 so on)

Column 3: File name (emp_01.ds ,emp_02.ds ,emp_03.ds )

Column 4:Invocation id (1,2,3,4........)

Please let me know this approach works for my requirement ??if possible I will post my sample "routine code"in next post

Thank you




Thank you

[ray.wurlod]

Definitely possible.

Code: Select all

OpenSeq file
Loop While ReadSeq Line from filevariable
   * parse line
   * attach job
   * set parameters checking for errors
   * submit job run request
   * when job finishes check job status
   * detach job
Repeat
CloseSeq filevariable