DS Log - One Job Loading Multiple Tables

gdean · Post by **gdean** » Thu Jun 24, 2004 12:38 pm

Hi,

I have a job that loads data from a flat file into 3 different Oracle tables(same columns in all 3 tables) depending on some conditions. For ex. if X =1 load into Tab1, if X = 2 load into Tab2, if X = 3 load into Tab3. To implement this I have one input link to a Transformer stage and then 3 output links to each one of the Oracle tables.

I have given the conditions in the constraints section of each link. The jobs runs fine as per the expectations. But, even when only one of the tables is loading there is an entry for each of the other tables as well in the log file (I know that can't be altered).

Eg. Log:

100 rows written to Tab1_insert
0 rows written to Tab2_insert
0 rows written to lTab3_insert

We load the audit tables with the info from the log files. Hence, the audit from the above log file reflects that actually 0 rows were written to Tab2 and Tab3. But this is faulty information(or is it not

) because the Tab2 and Tab3 were not loaded logically.

How can I make sure that these extra log records are not loaded into the audit tables? We have a routine for reading the log file and it is used by all other jobs as well and hence cannot be changed. Is there a better way to implement what I'm trying to achieve?

Thanks to you all,
Gregg

chucksmith · Post by **chucksmith** » Thu Jun 24, 2004 1:17 pm

Do all three tables have the same record layouy? If so, you could pass the table name as a job parameter, and your job would have only one output link. Otherwise, you may have to look at your routine that gathers stats.

gdean · Post by **gdean** » Thu Jun 24, 2004 1:34 pm

Sorry for not providing that information earlier.

All the three tables have the same layout. But the update action is different for each of the tables. Hence the need for three seperate output links.

Thank You,
Gregg

kduke · Post by **kduke** » Thu Jun 24, 2004 2:25 pm

Gregg

I am not sure I get why you have a problem. The log shows all records going to the first link? If so then try DSGetJobInfo, DSGetStageInfo and DSGetLinkInfo. Do a search. Several people have posted accurate ways of getting row counts. If you cannot find the posts then let me know.

chucksmith · Post by **chucksmith** » Thu Jun 24, 2004 4:46 pm

How different are your update actions? Could they be hidden behind user defined SQL? Maybe, you should just go to three jobs with one link each.

gdean · Post by **gdean** » Mon Jun 28, 2004 2:59 pm

Hi,

Sorry for the delay in getting back to you. I have finally divided the one job into 3 seperate jobs each loading a different table. That seemed to be the straight forward way to solve my problem of the log info.

Thank you Chuck and Kim for your attention.

-Gregg