Dynamic Schema Question ?

psriva · Post by **psriva** » Sat Aug 12, 2006 10:35 am

We have several processes written in AbInitio that we would like to port to the Grid using DataStage. The AbInitio jobs use dynamic DMLs (layouts) - the input & output layouts are generated automatically prior to the run, then the Abinitio job reads in the input/output DML (layout) files and processes the data using that format. Hence, without touching the
AbInitio code, we can process data with differing input and output layouts for each run.

Can this be done in DataStage?

chulett · Post by **chulett** » Sat Aug 12, 2006 1:58 pm

Didn't you already ask this question?

meena · Post by **meena** » Sat Aug 12, 2006 3:25 pm

Yes..the post is "Migration from Abinitio to Datastage"

The question frame is different....but the answer is going to be the same "RCP"...

psriva · Post by **psriva** » Tue Aug 15, 2006 9:17 am

Dear Friends,

The question is different and so is the post. The earlier question was about mapping to be done automatically and this question is about dynamic generation of layouts.
I have tried experimenting with a friend of mine and have found an answer to this.
When layouts are dynamically generated, we go to the input or the output stage and click on the options and select a schema file parameter (from the options) and provide a value for it ( which is the schema file location).
When we do this we dont have to load the columns within the columns tab and we dont have to do anything within the format tab. The columns are loaded dynamically and are propagated(as we specify the RCP).

ray.wurlod · Post by **ray.wurlod** » Tue Aug 15, 2006 10:40 pm

Have you given any thought to lineage analysis in all of this? That is, when an end user asks "where did this come from?" will you be able to supply an answer if all the schemas are being created dynamically? It is for this reason I try to avoid RCP.

jking123 · Post by **jking123** » Mon Aug 21, 2006 10:52 am

RCP in datastage is very limited. What you are asking you will have to be done with transformers and/or modify stages or with custom operators using osh but even then it may not be the same and you may end up with a custom job per source-target. In DS it is easier to hardcode than do highly parameterized jobs. I have tried replicating that functionality and if you find a good easy to maintain solution let me know.

Ab Initio has a great way of doing lineage accross dynamic metadata and esp with parameterized jobs where column names are only defined at run time. Currently DS can't even do lineage when using schema files and RCP. The metadata capablities are severley limited.

gsherry1 · Post by **gsherry1** » Mon Aug 21, 2006 12:53 pm

The dynamic metadata functionalities of Ab Initio jobs are far superior to DataStage. I'd recommend sticking with Ab Initio.

The only advantages of migrating to DataStage would be to get a more colorful canvas, reduced licensing cost, and to make use of the dsxchange forum

.

- Greg

jking123 · Post by **jking123** » Mon Aug 21, 2006 2:58 pm

The two migrations I have seen are reduced licensing cost. For the most simplest of AB Inito jobs the migration can be done cost effectively. For anything which use things like dynamic metadata, conditional
components, scans, continuous, xmls etc etc the migration cost is high, but depends on the number of things to convert.

It is MUCH easer to migrate to Ab Initio from Datastage as it has many more features. Licensing costs and cost of resources are high though.

miwinter · Post by **miwinter** » Tue Jan 23, 2007 8:50 am

... and the Ab Initio "GUI" is an utter disgrace....

/rant