Warnings from Match Frequency Stage

[sigma]

Hi

This is really not a warning but an informational message

When my data flows through the match frequency stage I get the message

Match_Frequency_17,0: 801763 input records read; 15657 kept

Just curious why does datastage read that much records and keep only 15k of them.

My input is about 32k rows. I expected my frequency output to be much higher then 32k as it has atleast 32k primary keys at a minimum

Am I missing something there. Why is the match frequency generating only 15657 rows especically when I say NONE for any match specificaiton in the match frequency stage. So it is based on all the columns.


Arvind

[vairus]

Hi arvind,

Match frequency give you information on how often a particular data appear and its format.

Example : you have customer_name column having data "john" for 150 times. Output give you the number of occurrence of john and statistical value in single row. So your output may be smaller than the input.

if your Match frequency output type is '.DS' dataset then dataset outputs are stored in \IBM\InformationServer\Server\Datasets

your output file contain only columns details and reference [where frequency data is stored] details.it will not be more than few kbs

Regards
Vairamuthu

Hi

This is really not a warning but an informational message

When my data flows through the match frequency stage I get the message

Match_Frequency_17,0: 801763 input records read; 15657 kept

Just curious why does datastage read that much records and keep only 15k of them.

My input is about 32k rows. I expected my frequency output to be much higher then 32k as it has atleast 32k primary keys at a minimum

Am I missing something there. Why is the match frequency generating only 15657 rows especically when I say NONE for any match specificaiton in the match frequency stage. So it is based on all the columns.


Arvind