QualityStage Job Error

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Tue Jun 30, 2009 12:20 pm

We are running into warning messages while running QualityStage jobs.

The exact warning message : External ustring too short. Imported only 0 external Characters into a ustring of fixed length 1. This is resulting in
thousands of records getting rejected.

We modified the Default field value from NULL to '' in a Sequencial File it helps in getting some of the rejected records through but still a significant number of records are in rejected state with the same error message.

Please help

Thanks

JRodriguez · Post by **JRodriguez** » Tue Jun 30, 2009 12:36 pm

Are those fields defined with char datatype? If yes just use varchar instead, that probably will help to avoid the warning messages

ray.wurlod · Post by **ray.wurlod** » Tue Jun 30, 2009 4:28 pm

If it's fixed length, then any default value must have the same number of characters as the field definition. Thus, for Char(1) - that is, ustring[1] - the default value "" is inappropriate - it must be " ".

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Tue Jun 30, 2009 8:13 pm

Hi,

Thanks a lot for your feedback. I tried changing all the fields with Char into Varchar it didnt help. All the records in the job were getting rejected with the same warning message.

I also tried changing default Null field value from '' to " " all the records were rejected with the same warning message

Another strange thing is that these jobs were running fine without any warnings previously 2 - 3 weeks back. However, now we are seeing this same warning message over and over again in multiple jobs with records getting rejected.

We did not apply any patch or make changes to these jobs

Thanks

ray.wurlod · Post by **ray.wurlod** » Tue Jun 30, 2009 10:37 pm

... which suggests that the data have changed. Do you still have older data that you can compare?

JRodriguez · Post by **JRodriguez** » Wed Jul 01, 2009 6:53 am

What's your value for APT_STRING_PADCHAR? If is 0x0(ASCII nulls) which is the default then try with 0x20(Space) and let us know

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Wed Jul 01, 2009 7:55 am

Hi,

We do have the previous jobs to compare against. Only thing that changed is, in the Extract job we changed the Null field value from NULL to empty string. We are suspecting that this is causing these warning messages in multiple subsequent jobs. We are running the extract job wihout any changes in Test environment to ensure that it is not causing the warning messages. I will update the post once it completes.

I couldnt APT_STRING_PADCHAR in the Administrative client under the Project. What is the significance of this?

Thanks

JRodriguez · Post by **JRodriguez** » Wed Jul 01, 2009 8:17 am

By default DS/QS used a null value(0x0) when extending or padding a string to a fixed length field (Char datatype), with this environment variable you can override the default value to any character. Most of the time you would like to use space(0x20)

You should find this env variable under Parallel category in the administrator tool

Did you try? let us know ...

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Wed Jul 01, 2009 11:57 am

Hi,

Thank you. I tried it. I changed the null value to 0X20. This helped get rid of few warning messages in one of the jobs we were having issue with but in the other job we are still getting the same warning and the records are rejected.

Thanks

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Wed Jul 01, 2009 12:04 pm

Hi,

The warning message is slightly different now
Following is the exact warning message:

Contact_AP_Dedup,0: External ustring too long. Imported only 2 out of 4 external characters into a ustring of fixed length 2. Ignoring last 2 characters of external ustring up to delimiter.

Thanks

JRodriguez · Post by **JRodriguez** » Wed Jul 01, 2009 12:11 pm

Did you type the metadata manually ... looks like you have different length .. now you have the inverse scenario... do a trim( ) on those fields having a warning or use same length in both input and output columns

Defining your column as Varchar avoid both scenarios ...

ray.wurlod · Post by **ray.wurlod** » Wed Jul 01, 2009 4:15 pm

Source data has four characters in a string; your metadata allows only for two. Change either, but probably the latter.

DataQuality_IS8.1 · Post by **DataQuality_IS8.1** » Thu Jul 02, 2009 7:57 am

Hi,

Our testing finished with jobs were changes were not made and this time there were no warnings. However, the only change we made was in the Extract job - Changed the Null handler in the Sequential file from NULL to '' which resulted in all these warning messages.

When we reverted the changes back to Null the job finished successfully with no records getting rejected in another environment

I am still not clear why this only change would cause records getting rejected. Can you please suggest any documentation that I can refer to understand how data fields are handled by QualityStage?

Our issue is resolved. Thanks a lot for your help

Thanks

DSXchange

QualityStage Job Error

QualityStage Job Error

Re: QualityStage Job Error

Re: QualityStage Job Error