i want to capture bad records

chandra · Post by **chandra** » Tue May 23, 2006 9:27 pm

how can i caputer the bad records from dataset.
suppose i have 10 records .
6th and 7th records are bad records .
how to achive this any clues?

kris007 · Post by **kris007** » Tue May 23, 2006 9:56 pm

It depends on how you define your bad records. Usually, you define a reject link to capture all reject data which are defined based upon the constraints as per your business requirements. In your case, from the example you have given,, it depends on how you identify your seventh and eighth records as bad data.

chandra · Post by **chandra** » Tue May 23, 2006 9:59 pm

i gave you example of 6th and 7th .
in this case i dnt want to use any constrains .
i want to capture the bad records from raw data.

ray.wurlod · Post by **ray.wurlod** » Tue May 23, 2006 11:02 pm

You don't use DataStage.

Constraints are DataStage's way of directing data flows.

To do it purely in source requires a HarryPotter stage.

vmcburney · Post by **vmcburney** » Tue May 23, 2006 11:14 pm

Have a look at my blog on the data quality firewall within a DataStage job. This lets you capture bad records as you process the raw data and deliver it to your datasets. If you want to identify bad records within the source databases you will need some type of profiling tool such as ProfileStage.

kumar_s · Post by **kumar_s** » Wed May 24, 2006 12:40 am

As mentioned, define what is 'bad record' in you case?
Is it alwasy 6th and 7th in order?

rony_daniel · Post by **rony_daniel** » Wed May 24, 2006 3:16 am

kumar_s wrote:As mentioned, define what is 'bad record' in you case?
Is it alwasy 6th and 7th in order?

Chandra,

Bad Records is purely a relative term. It depends on your business requirements. The record which one business calls as bad record may not be a bad record for another. Some of the common scenariaos would be like :-
1) If your source is a fixed length file (say 100) and if the 6th and 7th records are having a length of 95 and 105, then you may call it as BAD.
2) If you are loading the data into a database where you have defined some of the columns as Date datatype and your 6th and 7th record has values in these fields as "30/Feb/2006" and "31/Apr/2006", then you may call it as BAD. Similarly for numeric datatypes...