General DS Usage with Dimension Tables

[admin]

Hi All,
Quick question.....When I last used DS I always updated dimension table information in place with inserts and updates because the files coming out of the transformer were different files when the key was found and when the key was not found. When the dimension record was not found we generated a new key and sent it down the insert link.

In a new job a colleague (non DS person) has suggested that we should produce one flat file of all records and load that. So I was wondering...Can DS do this easily on WIN2000?

When I was on NT 4.0 I was told it was difficult in DS to take two files (one of updates and one of inserts) and concatenate them together to be able to reload a table using a load utility inside the DS job. Has anyone built dimension processing jobs that merge the inserts/updates so they can be reloaded?? Or merge any files so they can be reloaded for that matter.

The only time I have seen anything close to this is one job where a HASH file was used as the working storage for dimension table and new rows were written to the HASH file which took care of the inserts and updates then the HASH file was loaded into the RDBMS.......in that case the job was very dependent on the HASH file.....which worried me.

Thanks

Peter Nolan
Data Warehousing Consultant
Mobile: +353 879 581 732
Fax: +44 870 029 1677