Loading the same data from many source databases

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
Luk
Participant
Posts: 133
Joined: Thu Dec 02, 2004 8:35 am
Location: Poland
Contact:

Loading the same data from many source databases

Post by Luk »

Hi!
I'm using ODBC stage to get data from MSSQL server. On this server there are many databases with the same strtucture (i.e. for many different companies).

Is there a simple way to use the same jobs to different data ODBC sources??
Is there some possibilty to change data sources in many jobs in one time??
I can of course change path to concrete database in ODBC source (leaving name of data source) but is this the best way, I doubt.

regards
LUK
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Look for DRS.

You can always use parameters in ODBC DSN.

The only affecting factor may be the naming conventions allowed and the delimiter char.s in the underlying dbms.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Can you not pass the Database instance, userid, and password into your job as parameters so that it will connect to the appropriate ODBC database at runtime?
Luk
Participant
Posts: 133
Joined: Thu Dec 02, 2004 8:35 am
Location: Poland
Contact:

Post by Luk »

Can you not pass the Database instance, userid, and password into your job as parameters so that it will connect to the appropriate ODBC database at runtime?
you have right yhat will be working:)

but second option is more tempting for me :) because I don't want to add parameters to every job which I have done.
You can always use parameters in ODBC DSN.
could you be more specific, is it possible to have 1 ODBC source (on Unix, in .odbc.ini file) that points to many databases?? and how can I achieve this??
LUK
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

If you do not wish to add parameters to your job(s) then you are limited to using your one ODBC connection; this would entail you editing & changing your DataStage "odbc.ini" and perhaps "UVODBC.INI" files to shift the actual connection location around.

Although ODBC and standard SQL is supposed to be both global and portable across platforms and databases in practice there are idiosyncracies and each database vendor has added a couple of enhanced features which, when used, cancel out any true portability. What I mean is that although you might think that the tables are identical across the Databases most often they are not 100% the same. But perhaps you are in luck!
Luk
Participant
Posts: 133
Joined: Thu Dec 02, 2004 8:35 am
Location: Poland
Contact:

Post by Luk »

table are identical in 100% because database is a part of special financial system and must be the same every time!!

I know that I can manualy change ODBC sources, but Sainath said sth. about parameters and I don't know what he had in his mind.

anyway thanks for help!
LUK
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

All he meant was that you could set up multiple ODBC sources in the configuration files and then specify the correct one for each run via a job parameter used in the DSN field.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

But then, why are the tables identical and present in different rdbms? Is it some kind of extract produced from various different sources?
Luk
Participant
Posts: 133
Joined: Thu Dec 02, 2004 8:35 am
Location: Poland
Contact:

Post by Luk »

All he meant was that you could set up multiple ODBC sources in the configuration files and then specify the correct one for each run via a job parameter used in the DSN field.
so, it is the same solution as Arnd gave !!
LUK
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Yup. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Luk
Participant
Posts: 133
Joined: Thu Dec 02, 2004 8:35 am
Location: Poland
Contact:

Post by Luk »

But then, why are the tables identical and present in different rdbms? Is it some kind of extract produced from various different sources?
I will give example:

MSSQL database1:
tableA(col1,col2...)
tableB(col1,col2...)

MSSQL database2:
tableA(col1,col2...)
tableB(col1,col2...)

databases have the same structure, tables in databases have the same structure!!

I am doing 1 job in which I'm extracting data from tableA and B from database1, transform and put data into target DB. Then I am using the same job (with different source database - this issue was the main subject of this post) to extract data from tablaA and B from database2 and put it into another target DB.

so, the purpose is: to have one job, and us it many times on different source and target DB!!

sory! I see that I wrote very long reply!
:D [/quote]
LUK
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

In that case, you are leading yourself into a situation of changing the configuration files.
Post Reply