Page 1 of 1

final file from integrity to datastage

Posted: Sun Nov 23, 2003 12:04 pm
by nag0143
After address scrubbing using in integrity.... my finalfile is TEST3
how do i get that file into my datastage job

Posted: Sun Nov 23, 2003 3:27 pm
by vmcburney
Please indicate which version of QualityStage and DataStage you are using. In version 7.0 you use the QualityStage plugin for DataStage, in this stage you select a QualityStage project and job, you select the QualityStage input table for that job and the output table. You import the definition of those tables into DataStage.

Alternatively you can let QualityStage run the job in standalone mode and then read the TEST3 file in DataStage as a fixed width sequential file stage. You lose all the benefits of integration doing it this way.

Investigation

Posted: Sun Nov 23, 2003 4:20 pm
by nag0143
Thanks
I am using 6.0 can you plz suggest me and moreover after scrubbing
the addresses,
0000+^D++^D>T I am getting in the output file and how can i remove
these and delimeters from the output file .....

nag

Posted: Sun Nov 23, 2003 4:46 pm
by ray.wurlod
The pattern characters were generated during the investigation phase, to allow you to see what kinds of tokens exist in your data. After all phases (investigation, standardization, matching and survivorship) all information typically has been carried through. All you need to do is to create another procedure to generate just the columns you require from that file into another file, which is the output from the procedure and which will be the input file for DataStage.

In DataStage you use a Sequential File stage to read that file as a fixed-width format text file. In fact, even without removing any pattern or other columns, you can still do this, and discard the unwanted columns in the first Transformer stage in your DataStage job.

To specify that a file is fixed width in the Sequential File stage, go to the Format tab and choose the fixed width option. In the column widths field, enter a comma-separated list of column widths, for example 7,10,1,12,30

Posted: Wed Nov 26, 2003 9:47 am
by nag0143
Instead of writing another procedure you can use extract feature available in integrity..

Posted: Thu Nov 27, 2003 7:07 am
by ray.wurlod
I'd recommend you re-think that statement. Typically the final step in INTEGRITY processing is a pre-built procedure, Survivorship. You can't add your own operators to pre-built procedures.
On the other hand, you can add pre-build procedures to your own user-defined procedures. But best practice is to keep them separate from each other, so that things are more easily maintained in the future.
By "extract" I assume mean an FFC operator, which cannot be performed in isolation, it has to be placed into a user-defined procedure so that its input and output, and the details of exactly how to convert the format, can be specified.