File & data level validations

[Narayana]

Hi,

I recently started working on DataStage PX and wanted to do file and data level validations. Please help by throwing some light on how to go ahead.

Regards,

Nari

[rachitha]

hi Narayana

Welcome aboard
What is the source and give the details of what validation you want to do.

[kumar_s]

Do you have any validation on the file name with Dates? As mentioned, you need to explain clearly on what exactly you looking for.

[Narayana]

my main aim is to know why we do validations in file level as i know checking the validation of the data for nulls, correct format, alpha characters in the numeric columns.

I hope it is clear.

[vmcburney]

There are implicit metadata validation and conversions that the stages perform without any code and explicit validation and conversion that you can encode into a Modify or Transformer stage via functions. Generally if data comes from a database most of it can be left as implicit - an integer field out of a database table will always be integer. If you have a free text field such as date or address you will want validation and conversion in one in a transformer to convert from character to date.

You could read every field in as varchar 255 and do validation and conversion on every single field but this is inefficient and time consuming and would only be required if the source data metadata was completely unreliable between each run.

[Narayana]

Thanks for your reply vmcburney...i could get good knowledge from your explanation about data validation.

but why do you we need file level validation and how do we do that using the stages in parallel extender.

[chulett]

As noted - perhaps you should define what you mean by 'file level validation'. Otherwise, people have to guess.

[Narayana]

how should i do that....

[ray.wurlod]

... are you the Moderator ?