Managing repository

dstgx08 · Post by **dstgx08** » Tue May 04, 2010 3:57 am

Hi,
I need to know how to manage repository Datastage v8.0.1
If I have usr/pwd to access to repository DB, I can simply do query to retrieve any information?
What kind of information I can retrieve?
Is there any other way to access to repository? (eg through command on shell Unix...dsadmin,dssh...etc)

regards
thanks

Michael

chulett · Post by **chulett** » Tue May 04, 2010 5:27 am

Query != Manage, which are you really asking about? And most people will have zero need to ever do any direct queries against XMETA.

dstgx08 · Post by **dstgx08** » Tue May 04, 2010 5:34 am

I'd like to query DB to retrieve information like:
job details, error log after job run..etc.

chulett · Post by **chulett** » Tue May 04, 2010 5:37 am

Why not just use the existing tools for that? That or the API if you really feel the need to 'code' something, job or script-wise. There's really no burning need for manual queries against the repository they purposefully obfuscate and leave unpublished.

That is, after all, exactly why those things exist.

dstgx08 · Post by **dstgx08** » Tue May 04, 2010 8:59 am

Wich tools/API do you refer?

tks

eostic · Post by **eostic** » Tue May 04, 2010 11:01 am

There are many... though for what you are noting here, dsjob is probably the most veratile, letting you dive deeply into job runs, link results, etc. etc. It comes in a variety of flavors, whether you prefer to do things in a shell at the OS command level, or C++, or directly from within DataStage.

It is well documented and their are tons of threads in this forum on the subject. Also check out ETLStats (search thru here and/or get out to Kim Duke's fine web site). It is an application that may already do a lot of what you are looking for.

Ernie

kduke · Post by **kduke** » Tue May 04, 2010 2:39 pm

EtlStats will extract simple metadata like job names and categories. There are jobs which can extract tablenames from PX jobs. It is a loose set of jobs to help get you started extracting metadata. The run time metadata is mostly what is used to extract job run times and row counts.

There are admin tools which help you backup your jobs every night with DOS batch files. There are other admin type scripts which can backup critical server files like the config files and the odbc setup files. These are critical if you lose a server or want to compare Dev to Test or Prod.

There are scripts for unlocking jobs which I may not have posted.

"Managing" could mean metadata or admin tasks. Maybe you need to provide us with more detail on what you are looking for.

kduke · Post by **kduke** » Tue May 04, 2010 3:03 pm

I have seen several questions about extracting the log information. This is not a task I would recommend. It can be one log record per record inbound. To extract that is not wise because when you have issues then these processes actually slow you down. I would look for specific things within the log. When we used Teradata at a previous site we would extract the Teradata log information. If you have Oracle then you could look for Oracle errors like ORA-9999. Something like that is very useful. Maybe you could keep track of how many time you had a specific Oracle error.

Be careful not to get too much metadata. Nobody will ever use it if you GBytes of log information.

dstgx08 · Post by **dstgx08** » Wed May 05, 2010 2:32 am

kduke wrote:I have seen several questions about extracting the log information. This is not a task I would recommend. It can be one log record per record inbound. To extract that is not wise because when you have issues then these processes actually slow you down. I would look for specific things within the log. When we used Teradata at a previous site we would extract the Teradata log information. If you have Oracle then you could look for Oracle errors like ORA-9999. Something like that is very useful. Maybe you could keep track of how many time you had a specific Oracle error.

Be careful not to get too much metadata. Nobody will ever use it if you GBytes of log information.

If I know how to access to repository (technology,ip,user,pwd), can I query it to retrieve some informations? (no matter what I get...)

chulett · Post by **chulett** » Wed May 05, 2010 6:32 am

Of course.

ray.wurlod · Post by **ray.wurlod** » Wed May 05, 2010 4:29 pm

Well, the repository is "open", so you can investigate in the same way that any of us can. Start, for example, by examining the system table that contains the names of tables in the XMETA database, then generate the column names, etc., for those tables that are of interest.