Comparing fields apart from usin change capture stage

[sdfasih]

Hi,
I would like to know is there any better way to compare two flat files(new and existing) containing 542 fields and upto 8000 records and tagging an update or new record character part from using change capture stage with only one primary key known,the reason for not using change capture stage is the time it gonna take for processing.
Thanx in advance.

[ray.wurlod]

Welcome aboard. :D

There are three stages for performing comparisons; you ought to investigate the Difference stage and the Compare stage. The Difference stage returns a simple result and all columns from both inputs; the Compare stage returns three columns; the result and the two input records as sub-records.

The Change Capture stage is really only for capturing the changes in a sub-record form that can be subsequently re-applied using a Change Apply stage.

[vmcburney]

I thought the difference stage was being phased out and replaced by the change capture stage. If you can identify a small set of key fields the change capture stage should have good performance, especially as it is running on parallel architecture. Most other methods will require a lot of field matching code.