No warning/error when data does not match schema definition

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
evee1
Premium Member
Premium Member
Posts: 96
Joined: Tue Oct 06, 2009 4:17 pm
Location: Melbourne, AU

No warning/error when data does not match schema definition

Post by evee1 »

My load job is a generic job that uses Column Import stage, a schema file (passed as a parameter) and RCP to load the data from a source file into the target table.
The test schema file contains the following definition:

Code: Select all

record { final_delim=end, delim='|', quote=none, null_field='' } (
abc:nullable string[max=4];
xyz:nullable string[max=120];
)
If a source file contains strings that exceed the size of the columns, my job jus happily truncates and loads the data.There is no error or warning reported.
Is it possible to make Datastage to complain when the source data does not match the schema, in case of trsings?
evee1
Premium Member
Premium Member
Posts: 96
Joined: Tue Oct 06, 2009 4:17 pm
Location: Melbourne, AU

Post by evee1 »

Also is there any documentation that would explain well what can be defined in the schema file, specifically record header and field options.
I search through online documentation, Red Books and documentation that comes with Datastage, but could not find any.
evee1
Premium Member
Premium Member
Posts: 96
Joined: Tue Oct 06, 2009 4:17 pm
Location: Melbourne, AU

Post by evee1 »

We have contacted IBM and we were advised to set the environment variable $APT_IMPORT_REJECT_STRING_FIELD_OVERRUNS to TRUE. This causes records with character strings not fitting the schema definition to be sent to the reject link.
Post Reply