Hi,
I would like to know is there any better way to compare two flat files(new and existing) containing 542 fields and upto 8000 records and tagging an update or new record character part from using change capture stage with only one primary key known,the reason for not using change capture stage is the time it gonna take for processing.
Thanx in advance.
Comparing fields apart from usin change capture stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Welcome aboard. :D
There are three stages for performing comparisons; you ought to investigate the Difference stage and the Compare stage. The Difference stage returns a simple result and all columns from both inputs; the Compare stage returns three columns; the result and the two input records as sub-records.
The Change Capture stage is really only for capturing the changes in a sub-record form that can be subsequently re-applied using a Change Apply stage.
There are three stages for performing comparisons; you ought to investigate the Difference stage and the Compare stage. The Difference stage returns a simple result and all columns from both inputs; the Compare stage returns three columns; the result and the two input records as sub-records.
The Change Capture stage is really only for capturing the changes in a sub-record form that can be subsequently re-applied using a Change Apply stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
I thought the difference stage was being phased out and replaced by the change capture stage. If you can identify a small set of key fields the change capture stage should have good performance, especially as it is running on parallel architecture. Most other methods will require a lot of field matching code.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn