Handling data truncation errors.

[Kryt0n]

If len(column) > 10 then reject... (syntax may not be correct)

[Pagadrai]

Hi,
What are your source and target stages in that job?

[ag_ram]

Hi Kryt0n,
This check has to be performed for all the columns and when the number of columns is more, this is going to be cumbersome. So this approach would not be suitable...Is there any way to identify from a system variable that a warning has occurred?

[ag_ram]

Hi,
Sources are predominantly DB Stages - Oracle and DB2. Targets could be files and/or DB Stages (Oracle and DB2).

[Pagadrai]

Hi,
Sources are predominantly DB Stages - Oracle and DB2. Targets could be files and/or DB Stages (Oracle and DB2).

Hi,
Actually am thinking if we can capture the records with length issue using a reject link for the target stages.
can you try that and let me know.

[Kryt0n]

Try the Column Import stage with a reject link.

Set the Length to the max length permitted and see if it drops any longer. If it doesn't reject the record when the length is greater, edit the column definitions and set the max length attribute... see if that works.

[shankar_ramanath]

Does this mean that the reject link does not handle format errors well?

I have a similar situation. I am using sequential file stage and have an input file with two columns. The sequential file stage has an output link to a transformer and a reject link. The format is as follows:

Field 1: VARCHAR(10)
Field 2: VARCHAR(30)

The input data contains 40 characters for Field 2. The execution results in the data being truncated for Field 2, retaining only the first 30 characters. I am somewhat baffled because this would mean that I would have to use a format similar to the following:

Field 1: VARCHAR(8000)
Field 2: VARCHAR(8000)

and check the length of the fields in the transformer stage in order to reject the record.

If I am not mistaken/presumptuous, that kind of defeats the purpose of the reject link.

The documentation is not very helpful. In the "Parallel Job Developer Guide", for reject link, I see

"For reading files, the link uses a single column called rejected containing raw data for columns rejected after reading because they do not match the schema."

Does this imply that data overflow is not considered part of the schema? At least this is not how it is in the database world

Thanks!

[Pagadrai]

Hi,
Is the issue resolved?
I just went through the following writeup in manual.
See if it helps your case.

" APT_IMPORT_REJECT_STRING_FIELD_OVERRUNS
When set, DataStage will reject any string or ustring fields read that
go over their fixed size. By default these records are truncated. "

Let me know once you try this.

[shankar_ramanath]

Hi Pagadrai,

Thank you for the input. Setting the value of this attribute to TRUE helped resolve the issue.

Thanks!