Page 1 of 1
Can i define metadata of file columns at run time.
Posted: Fri May 19, 2006 3:29 am
by anshumangupta1206
Hi All,
I don't know whether this is possible or not , but i want to define the columns metadata at run time. My exact requirement is, i have a meta data stored in one sequential file as a data , i want this metadata should applied to other sequential file (or any other) at run time i.e the structure of new file will be decided at run time.
Thanks,
Ansh
Posted: Fri May 19, 2006 4:42 am
by ray.wurlod
Not possible out of the box. DataStage is driven by metadata. The metadata must be in place first.
Posted: Fri May 19, 2006 5:08 am
by anshumangupta1206
Hi ray,
Thanks for the guidance :D .
I am thinking if I define one column with the arbitary length say 100, read all columns data into this and picks up data from the start and end length for a particular field if required. I know for many stages processing we have to define all columns seprately, but I just want a guidance that can we think on this side if at all this is possible.
Thanks,
Ansh
ray.wurlod wrote:Not possible out of the box. DataStage is driven by metadata. The metadata must be in place first. ...
Posted: Sat May 20, 2006 1:58 am
by ray.wurlod
Of course it's possible, except that you must have a priori knowledge of the metadata to be able to design it! This doesn't just appear out of thin air!
Posted: Sat May 20, 2006 6:01 am
by kumar_s
"i have a meta data stored in one sequential file as a data"
If this is the case you can make use of scheme file optiong to acheive a job to get metadata from the file.
Meta data stored in sequential file.
Posted: Tue May 30, 2006 4:09 am
by anshumangupta1206
Kumar,
Can u through some more light what is this scheme file option.
kumar_s wrote:"i have a meta data stored in one sequential file as a data"
If this is the case you can make use of scheme file optiong to acheive a job to get metadata from the file.
Posted: Tue May 30, 2006 5:36 am
by kumar_s
In sequential file, under Options you can find Schema file. You can give the path and name of the file which holds the metadata model for the stage.
A sample structure would be
Code: Select all
record (
name:string[255];
address:nullable string[255];
value1:int32;
value2:int32
date:date)
If you have enabled RPC (Run time column propagation), you can edit the schema file externally and can find the impact in the job without editing it.
You can find the informations under Parallel job developers guide.
Can i define metadata of file columns at run time.
Posted: Tue May 30, 2006 5:55 am
by ashwin141
Hi Anshuman
Well its possible if you are having rough idea about the source metadata. We did it using a Generic stage and Orchestrate operators. Though its a complex way of doing it. It can be done. But from I will suggest you should avoid this Metadata driven approach.
Regards
Ashwin
Posted: Tue May 30, 2006 6:54 am
by chulett
So now suddenly we are giving PX solutions to Server problems?
Posted: Tue May 30, 2006 7:07 am
by ray.wurlod
Seems that way!
Can i define metadata of file columns at run time.
Posted: Tue May 30, 2006 8:11 am
by ashwin141
I am sorry about that, didn't see the job type
My Apologies.
Ashwin