rejected records

sundar · Post by **sundar** » Thu Jul 27, 2006 3:07 am

Hi All

From source 83 records coming to transformer
but out of the transformer only 2 records are coming,no records were rejected.

I am getting the following warning

APT_CombinedOperatorController(3),1: Field 'pt_state' from input dataset '0' is NULL. Record dropped

APT_CombinedOperatorController(3),1: Warning, all other rejected records will be silent.

anyone help me in this

Thanks
sundar

ArndW · Post by **ArndW** » Thu Jul 27, 2006 3:26 am

Sundar,

the warning is telling you that at least one record and more likely all 81 missing rows are being dropped because they have a NULL value and your target column does not allow nulls.

sundar · Post by **sundar** » Thu Jul 27, 2006 3:42 am

Hi Arndw,

Thank u for u'r reply

my Target columns are nullable.

i have one column which is char(1) in source
char(2) in target
so i concatenate with " " to make char(2)

will cause the problem

please help me

thanks
sundar

raghudsx · Post by **raghudsx** » Thu Jul 27, 2006 3:57 am

Hi,

U need not concatinate with any thing to load the char1 field in char2 field of your target.

What are your source and targets? Seq files?

Regards

Raghu

sundar · Post by **sundar** » Thu Jul 27, 2006 4:06 am

Hi raghu

thank u
my job design look this.i am applying rules in the transformer.

dataset--->join---->transformer----->seq
|
dataset

thanks
sundar

balajisr · Post by **balajisr** » Thu Jul 27, 2006 4:12 am

From source 83 records coming to transformer
but out of the transformer only 2 records are coming,no records were rejected.

I am getting the following warning

What is special about those 2 records?

ArndW · Post by **ArndW** » Thu Jul 27, 2006 4:25 am

I can state that the difference in these two is that column "'pt_state" is not null.

thompsonp · Post by **thompsonp** » Thu Jul 27, 2006 4:54 am

Are you using any of the fields that contain nulls in stage variables? You need to specifically code for nulls in stage variables to avoid records being dropped.

iamnagus · Post by **iamnagus** » Thu Jul 27, 2006 5:50 am

If any nullable field being used in the Transformer, need to handle that field. In this scenario use NullToZero() function for the field 'pt_state" then make Nullable 'NO'. It works for you. Check.

kumar_s · Post by **kumar_s** » Thu Jul 27, 2006 7:09 pm

Field 'pt_state' from input dataset '0' is NULL. Record dropped

Reconfirm the output metadata of the transformer for pt_state column is nullable.
If you source is nullable = 'no', check the RCP. Disable it.
Esle change this column as nullable = yes right from begining and do a 'Force compile' and rerun the job.

alphac · Post by **alphac** » Thu Jul 27, 2006 7:55 pm

You may also check the "null field value" property set in "Sequential file stage".

In your case, the transformer / the sequential file would drop records if :
1. transformer : put a "NULL" to a non-nullable field
OR
2. sequential file : you have not set "null field value" for stage to handle how it represent "NULL" on IO disk.

you may check it also.

p.s. "null field value" property is located in
right click row -> edit row -> click "nullable" -> you may find "null field value" in property window.

Hope it can help you.

mctny · Post by **mctny** » Thu Jul 27, 2006 11:14 pm

Transformers can handle null values without any problem, you don't need to use any null field value etc, however sequential files cannot handle nulls by itself, you need to handle by yourself. ( use special null value as suggested, )

kumar_s · Post by **kumar_s** » Fri Jul 28, 2006 1:16 am

Hi Cetin,
Have you treid assigning a Null to a not null field in a transformer?

thompsonp · Post by **thompsonp** » Fri Jul 28, 2006 1:35 am

Cetin,

Whilst you're trying kumar's test you might also want to add a stage variable and try using a field containing a null value in the derivation e.g. testing if it is equal to some other field.

vmcburney · Post by **vmcburney** » Sat Jul 29, 2006 3:36 pm

This is quite common when you put a transformer after a join or lookup stage. You end up running a transformation function against a reference field that can be null.

You need to put null handling around any transformer field that might contain a null and requires some type of transformation change or checking. Especially the columns that have come back from the join.

You need a reject link from the transformer to trap those rows that are getting dropped off. You might fix this first NULL problem but a new problem may show up weeks later.