implict conversion in sequential stage

[Karine]

I'm testing implicit conversion in sequential file stage. The seq file has single column with 7 rows:

1
aaa
3333
4A
4B8
345.7
2006-10-20

My output definition for col1 is Integer as I want any non integer data to be sent to the reject link.

However when I view the data in data browser, I'm getting the following rows :

1
3333
4
4
345
2006

I'm totally perplexed as I would expect the original rows 2,4,,5,6,7 would be rejected by implicit conversion and should only return

1
3333.

can someone please help my understanding why this is happening and how could I achieve my aim.

Karine

[ray.wurlod]

It is obvious WHAT is happening, WHY is far less apparent. What is the field delimiter, quote and final delimiter character specified? Please also advise the record delimiter property value. Then we can try to reproduce.

[Karine]

Thanks Ray, The following are my settings:

Record Level Field Delimiter = end, Record delimiter (not specified)
Field Defaults - Delimiter=none, Quote=none

[kumar_s]

What is the metadata specified in your sequential file stage.
Since Field Defaults - Delimiter=none, it is reading only the numeric values that existing in the field.

[ray.wurlod]

Now we need to know the data type specified for your single field.

I would imagine, as you did, that if it's integer then the rows containing non-numeric characters should be dropped or sent along a reject-handling link.

I have reproduced this, it also occurs with the record delimiter property set to "UNIX newline" both with and without the final delimiter property. I think you've uncovered a bug, which you need to report through your support provider.

Meanwhile, as a workaround, import the field as a VarChar and test its integer quality in a Transformer stage using the IsValid() function.

Code: Select all

IsValid("int32",InLink.TheField)