Can i define metadata of file columns at run time.

anshumangupta1206 · Post by **anshumangupta1206** » Fri May 19, 2006 3:29 am

Hi All,

I don't know whether this is possible or not , but i want to define the columns metadata at run time. My exact requirement is, i have a meta data stored in one sequential file as a data , i want this metadata should applied to other sequential file (or any other) at run time i.e the structure of new file will be decided at run time.

Thanks,
Ansh

ray.wurlod · Post by **ray.wurlod** » Fri May 19, 2006 4:42 am

Not possible out of the box. DataStage is driven by metadata. The metadata must be in place first.

anshumangupta1206 · Post by **anshumangupta1206** » Fri May 19, 2006 5:08 am

Hi ray,
Thanks for the guidance :D .

I am thinking if I define one column with the arbitary length say 100, read all columns data into this and picks up data from the start and end length for a particular field if required. I know for many stages processing we have to define all columns seprately, but I just want a guidance that can we think on this side if at all this is possible.

Thanks,
Ansh

ray.wurlod wrote:Not possible out of the box. DataStage is driven by metadata. The metadata must be in place first. ...

ray.wurlod · Post by **ray.wurlod** » Sat May 20, 2006 1:58 am

Of course it's possible, except that you must have a priori knowledge of the metadata to be able to design it! This doesn't just appear out of thin air!

kumar_s · Post by **kumar_s** » Sat May 20, 2006 6:01 am

"i have a meta data stored in one sequential file as a data"
If this is the case you can make use of scheme file optiong to acheive a job to get metadata from the file.

anshumangupta1206 · Post by **anshumangupta1206** » Tue May 30, 2006 4:09 am

Kumar,
Can u through some more light what is this scheme file option.

kumar_s wrote:"i have a meta data stored in one sequential file as a data"
If this is the case you can make use of scheme file optiong to acheive a job to get metadata from the file.

kumar_s · Post by **kumar_s** » Tue May 30, 2006 5:36 am

In sequential file, under Options you can find Schema file. You can give the path and name of the file which holds the metadata model for the stage.
A sample structure would be

Code: Select all

record (
name:string[255];
address:nullable string[255];
value1:int32;
value2:int32
date:date)

If you have enabled RPC (Run time column propagation), you can edit the schema file externally and can find the impact in the job without editing it.
You can find the informations under Parallel job developers guide.

ashwin141 · Post by **ashwin141** » Tue May 30, 2006 5:55 am

Hi Anshuman

Well its possible if you are having rough idea about the source metadata. We did it using a Generic stage and Orchestrate operators. Though its a complex way of doing it. It can be done. But from I will suggest you should avoid this Metadata driven approach.

Regards
Ashwin

chulett · Post by **chulett** » Tue May 30, 2006 6:54 am

So now suddenly we are giving PX solutions to Server problems?

ray.wurlod · Post by **ray.wurlod** » Tue May 30, 2006 7:07 am

Seems that way!

ashwin141 · Post by **ashwin141** » Tue May 30, 2006 8:11 am

I am sorry about that, didn't see the job type

My Apologies.

Ashwin

DSXchange