Metadata Table Definitions: thoughts

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
luca
Participant
Posts: 81
Joined: Wed May 14, 2003 11:34 pm

Metadata Table Definitions: thoughts

Post by luca »

Hi there,

I have been trying to find an efficient convention to store the metadata using DS designer and manager (stages, links and file name naming convention in the designer and convention related to the way of saving metadata from the designer or manager).

When trying to make a few changes using the manager, I once ended up with this message:
***
Category '......' must contain exactly 2 non-empty fields separated by ''.
Field 1 is the data source type (for example: 'ODBC', 'Hashed', 'Sequential', 'COBOL FD').
Field 2 is the data source name.
***

This could be fine, but it seems that it is not always working this way, depending on the type of the data source.

ex: when saving metadata using the ORAOCI plug-in, DS defaults "PlugInORAOCI9" to store the DB tables definitions. Here it doesn't match the pattern CategorySubcategory= "DataSourceTypeDataSourceName".
Would it be a good idea to change the DS defaults to match the pattern "DataSourceTypeDataSourceName"? Do you have in mind some reason not to do so?

Moreover, following are the defaults that DS suggests when I want to save table definitions from a job (from the designer):

1/ Sequential File:
Data Source Type = Saved (which I replace by Sequential)
Data Source Name = Sequential_File_0 (name of the stage)
Table/file name =

2/ Hashed File:
Data Source Type = Saved (which I replace by Hashed)
Data Source Name = Hashed_File_2 (name of the stage)
Table/file name = DSLink4

Note (for 1/ and 2/): the file name suggested by DS by default is empty for a source but it defaults the link name for a target. Is there a special reason for that? Has someone seen an interest in this? How do you deal with the defaults, when do you keep it and why? and when do you replace, by what and why?

What do you think would be a good choice for the Data Source Name for sequential and hashed files?

I have defined a convention to partition all the jobs by category. I wanted to do the same kind of thing for the "table definitions" to allow for example to do an export for both the jobs and the "table definitions" by category (export the jobs in a specific category and the table definitions related). Is there an other way to do something like this?

Have you been trying to go through such a convention, what problems did you face? do you have some tips or suggestions?
When saving metadata, when do you think it is preferable to keep the DS defaults and when do you prefer to replace by your own choice?
Would some changes have unfortunate consequences (ex: if using MetaStage)?
Did someone manage to match the pattern "DataSourceTypeDataSourceName" for all the data source type? If so, can you tell about what DS defaults you had to change.

I will appreciate any suggestion, answer or thoughts that this topic might inspires you.

Cheers,

Luca
Post Reply