Comparing two tables with same metadata
Moderators: chulett, rschirm, roy
Comparing two tables with same metadata
Hi Everyone
We are trying to compare two tables with same structure from different schemas. We have about 300 tables in both schemas. Instead of developing separate jobs, we want to build a single common job with run time column propagation on and loop the table names through the sequencer.
I have built a parallel job,
Oracle connector stage from both the schemas as input - Difference stage - Target table (differences are loaded)
The challenge I am facing here is configuring the difference stage, Difference stage is expecting key column and if create a single job how can I update the key column dynamically in difference stage (as key column varies from table to table)?
Or do I need to use any other stage so that I can reuse the same common job? Please suggest any best possible solution to achieve this.
Thank you all in advance
We are trying to compare two tables with same structure from different schemas. We have about 300 tables in both schemas. Instead of developing separate jobs, we want to build a single common job with run time column propagation on and loop the table names through the sequencer.
I have built a parallel job,
Oracle connector stage from both the schemas as input - Difference stage - Target table (differences are loaded)
The challenge I am facing here is configuring the difference stage, Difference stage is expecting key column and if create a single job how can I update the key column dynamically in difference stage (as key column varies from table to table)?
Or do I need to use any other stage so that I can reuse the same common job? Please suggest any best possible solution to achieve this.
Thank you all in advance
thank you
- prasad
- prasad
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What do you mean by "escape" the key column?
The only non-custom stage of which I'm aware that can use a parameter as an input column name is the Modify stage.
The only non-custom stage of which I'm aware that can use a parameter as an input column name is the Modify stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Use a Diff-operator in a generic-stage.
In addition, You will have to use preceding hash- and tsort-operators to sort data correctly and handle partitioning for multiple nodes.
Use the same (parameterized) key-settings for hash, tsort and diff-operators. Find additional parameterization options for the operators in the Parallel Job Advanced Developer's Guide.
In addition, You will have to use preceding hash- and tsort-operators to sort data correctly and handle partitioning for multiple nodes.
Use the same (parameterized) key-settings for hash, tsort and diff-operators. Find additional parameterization options for the operators in the Parallel Job Advanced Developer's Guide.
"It is not the lucky ones are grateful.
There are the grateful those are happy." Francis Bacon
There are the grateful those are happy." Francis Bacon