Page 1 of 1

100 columns only need to scrub five

Posted: Tue Nov 18, 2003 3:03 pm
by ray.wurlod
Posted on behalf of Nag, who sent a private message. :oops:

hello Ray,
I have a rawdata and datafiles i need to do address scrubbing on five columns out of 100 columns in the raw data. What is the procedure should i follow and should i keep the raw data in the data/projects directory

as i am new to this forum i don't know how do send to all users.
thanks
nag

Posted: Tue Nov 18, 2003 3:06 pm
by ray.wurlod
Create the following initial procedures.
ADDKEY add a unique identifier to each row (can be a sequence)
STORE95 put the 95 unneeded columns into a file, with the key
STORE5 put the 5 needed columns into a file, with the key

The STORE5 output file becomes the input file for scrubbing. Preserce the key values added by the ADDKEY procedure.

Create a final UNIjoin procedure to re-associate the other 95 columns with your scrubbed data.