File & data level validations
Moderators: chulett, rschirm, roy
File & data level validations
Hi,
I recently started working on DataStage PX and wanted to do file and data level validations. Please help by throwing some light on how to go ahead.
Regards,
Nari
I recently started working on DataStage PX and wanted to do file and data level validations. Please help by throwing some light on how to go ahead.
Regards,
Nari
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
There are implicit metadata validation and conversions that the stages perform without any code and explicit validation and conversion that you can encode into a Modify or Transformer stage via functions. Generally if data comes from a database most of it can be left as implicit - an integer field out of a database table will always be integer. If you have a free text field such as date or address you will want validation and conversion in one in a transformer to convert from character to date.
You could read every field in as varchar 255 and do validation and conversion on every single field but this is inefficient and time consuming and would only be required if the source data metadata was completely unreliable between each run.
You could read every field in as varchar 255 and do validation and conversion on every single field but this is inefficient and time consuming and would only be required if the source data metadata was completely unreliable between each run.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: