Generic job

srinivas.g · Post by **srinivas.g** » Mon Apr 27, 2009 12:33 am

Hi All,

I am trying to create a generic job which will load from any table regardless underlying metadata. What it means is,
Job should load even if there are different numbers of columns
Different number of tables and for difference where conditions

My source and target are oracle database.

I am trying both in server and parallel jobs. Mostly I wanted to do this implementation in server jobs.

Is there any work around to achieve this server jobs. Pls suggest

Prakash

mahadev.v · Post by **mahadev.v** » Mon Apr 27, 2009 12:55 am

First thing that comes to my mind in a parallel job is using RCP. Any specific reason to prefer server jobs? Is there an equivalent property to RCP in server jobs?

ray.wurlod · Post by **ray.wurlod** » Mon Apr 27, 2009 2:24 am

There isn't. DataStage is driven by metadata, not vice versa.

Even RCP means all you're doing is EL, not ETL. If you're going to do transformation, you need to name the source column(s) explicitly.

And there are cheaper tools for doing EL than DataStage.

sbass1 · Post by **sbass1** » Mon Apr 27, 2009 9:33 pm

Hi,

Any suggestions for EL tools "out there"? Freeware would be great, otherwise cheap. Something that would just copy data from ODBC Source A to ODBC Source B, without any transformation, that would be quick and easy to use?

We often have need to just do a "bulk load across databases" (EL), with further manipulation (ETL) after the bulk load. I was just wondering if such a tool could augment the functionality in DS?

This is somewhat related to viewtopic.php?t=126685&highlight=

Thanks,
Scott

ray.wurlod · Post by **ray.wurlod** » Mon Apr 27, 2009 10:37 pm

Most database vendors have their own. The Red Brick copy management utility is a particularly brilliant example of its kind.

I don't know, never having searched, whether there's a generic ODBC to ODBC product out there. Has anyone searched for one? DTS, maybe?

srinivas.g · Post by **srinivas.g** » Tue Apr 28, 2009 5:21 am

ray.wurlod wrote:There isn't. DataStage is driven by metadata, not vice versa.

Even RCP means all you're doing is EL, not ETL. If you're going to do transformation, you need to name the source column(s) explicitl ...

I understand the use of the RCP but could you please help me with some workaround in the parallel itself to do the transformations.

chulett · Post by **chulett** » Tue Apr 28, 2009 6:39 am

Meaning, you want to build a "generic" PX job with RCP that has no columns defined in it and still do transformations on the data? No can do.