Dynamic handling metadata

[rivajtp]

HI,

I have a job(suppose for Job A) which take data from input file(Sequential file stage) and load into target table(DB2 Stage) in between a tranformer which does some validation.

I have around 100 table which has the same type of tranformation and job design. but 100 table has different metadata .

Is there any way i can create only one job and dynamically it maps the metadata data (Squential file stage and DB2 STage) .

In another words one job which does the load for all the table which has similar tranformation put in the tranformer to reduce my developement and maintance effort.

Can any one please comment on the idea and give inputs

Regards
Rivaj T P

[ray.wurlod]

Welcome aboard. :D

There is no way to achieve this in a server job.

[rivajtp]

HI Ray,

Thanks for the reply.

Keeping in mind my final goal of loading the table for various metadata. i relax my gateway now from having one job to 100 jobs for 100 table, but to reduce the developement time and hence cost. Is it possible for me to write a program which take the template (Job A developed) and create other job for Other metadata

Regards

[ag_ram]

I have around 100 table which has the same type of tranformation and job design. but 100 table has different metadata .

I have one question:

How the same type of transformation would be applicable to different metadata of source?

[ag_ram]

I have around 100 table which has the same type of tranformation and job design. but 100 table has different metadata .

I have one question:

How the same type of transformation would be applicable to different metadata of source?

[rivajtp]

Same type of tranformation which i meant is there are some validation done on key column, key generation, etc which is identitical. columns are with different metadata but one to one mapping from source to target

[ag_ram]

Though, i am not clear with your transformation as having questions - what about Input metadata before and after transformation, is it possible to seperate light weight transformation in order to make same input and output metadata for main transformation.

1. Seperate Extract, Transform and Load as Jobs.

2. Reuse Transform and Load components.

3. (You may) keep light weight transformation in Source stage(Source Job).

[ArndW]

The most common method to do this kind of work would be to write a template example job, then do a DataStage export to either XML or .ds format and write a job or a program to read this file and change those elements around that are different and create 100 jobs this way.

[rivajtp]

HI,

that is a good solution.
1) which language u will prefer to write the outside program
2) while importing back if any bug. i feel is that there is some chance of getting corrupting the project. what you say on that?

Please comment on above question

Regards
Rivaj t P

[rivajtp]

HI,

that is a good solution.
1) which language u will prefer to write the outside program
2) while importing back if any bug. i feel is that there is some chance of getting corrupting the project. what you say on that?

Please comment on above question

Regards
Rivaj t P

[ray.wurlod]

I believe U would prefer to use DataStage BASIC, since U is currently working at a server-only shop in Singapore (or was, last time I looked).

[dilip.datastage]

Hi Rivaj,
As ArndW said we can use template and create 100 jobs using a job or program.

Hi Andrw, can you suggest how to proceed on. Has any one implimented same before. Me and Rivaj are working for same problem? If there is an metadata change after generating 100 jobs how can we go and chage again....

[rivajtp]

HI Ray,

Doing export and doing is good solution.

for my understading,

can i achive in parallel extender, if yes what is the lacking in server where i am failing to achive in server job. (Little bit to internal of datastage)

Regards

[ray.wurlod]

There should be no difference in the process based upon job type.

Only design-time components are exported - parallelism is not determined until run-time in parallel jobs.