where are the log files for DS

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
hiltsmi
Participant
Posts: 20
Joined: Thu Aug 04, 2005 9:03 am

where are the log files for DS

Post by hiltsmi »

I know how to use Director to view job logs. I would like to view/use the job logs outside of the director.

Where does DS store job logs? Are they in a text file that can be edited?
djm
Participant
Posts: 68
Joined: Wed Mar 02, 2005 3:42 am
Location: N.Z.

Post by djm »

Try this thread: viewtopic.php?t=95633.

David
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: where are the log files for DS

Post by chulett »

hiltsmi wrote:Where does DS store job logs? Are they in a text file that can be edited?
David seems to have thrown you straight into the Advanced Placement class when it doesn't even seem like you've taken the 101 version yet. :wink:

In addition to all that, the simple answer is 'no' - they are not text files. You'll soon find that the vast majority of information stored in the repository is stored in hashed files just like you would use in a job, and the logs are no exception. Each job is assigned a number when it is created and a series of hashed files are created to hold its design, object code, logs, etc. Have a peek inside one of your Project directories and you'll see.
-craig

"You can never have too many knives" -- Logan Nine Fingers
djm
Participant
Posts: 68
Joined: Wed Mar 02, 2005 3:42 am
Location: N.Z.

Post by djm »

But Craig, my reply was (like my forum moniker) short and to the point :wink:
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

<musing>
I note that we have a poster called ak77. I wondered if there's an ak47.
</musing>
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

The newer model is the ak74. Played with one of those, the ak47 is better.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

D wrote:But Craig, my reply was (like my forum moniker) short and to the point :wink:
Yes, that it was. :D

Not sure what Ray and Arnd are prattling on about, however...
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Does the word "Kalashnikov" rattle any eardrums?

I fear we're drifting off topic again.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Oh, perfectly well aware of what a Kalashnikov AK47 is - just wasn't quite sure of the relevance to the topic being discussed.

And as we drift ever further from the dock... I guess the connection was short usernames. Not nice to poke fun at 'short' people. And you know Arnd has a soft spot for automatic weapons. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

You guys are on another wave length.
:shock:
Mamu Kim
htrisakti3
Charter Member
Charter Member
Posts: 36
Joined: Thu Jun 10, 2004 11:22 pm

etlstat cannot be deleted

Post by htrisakti3 »

hi KDuke,
I downloaded etlstat & imported into DS but now I cannot delete them.. ?

This solution does not work for me, because we have very limited access to DB, therefore I'm still searching for solutions based on hash or files only (anything that's only limited to ds, w/o touching DB)'

Thanks - HT
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Use the dsjob API to dump log summaries. If you want full log dumps, youre going to have to write something using intenal knowledge of the job repository. The APIs are not complete enough to give you full dumps of the job log.

The simplest thing to do is cross-reference the job name to the job number out of DS_JOBS. Then, dump out the hash file RT_LOGnnn where nnn is the job number. You'll have to learn DS BASIC and figure out how to sort and normalize the data.

One thing to consider is writing a DS job to point at the RT_LOGnnn file using the hash file stage and stream the output to a sequential file. The hash file will have to be a job parameter that can change from one run to another. That will allow you to more easily parse that log file. At least poke around and see what you can come up with.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

hi KDuke,
I downloaded etlstat & imported into DS but now I cannot delete them.. ?

This solution does not work for me, because we have very limited access to DB, therefore I'm still searching for solutions based on hash or files only (anything that's only limited to ds, w/o touching DB)'

Thanks - HT
You can run this against hash files. You can change all the stage types to hash files in job DSJobReportDb. Also why can't you delete these jobs? When you use hash files the DSN is localuv. Leave the user and password blank for localuv. The job GenHtmlJobLog uses SQL to extract the log. This could easily be changed into a UV stage and extract the RT_LOGnnn file directly. You would need to change the nnn into a job parameter like RT_LOG#JobNo#. Anyway here are the SQL statements from the job above.

Code: Select all

      SqlStmt = ''
      SqlStmt< 1> = 'select'
      SqlStmt< 2> = '   TIMESTAMP,'
      SqlStmt< 3> = '   SEVERITY,'
      SqlStmt< 4> = '   FULL.TEXT'
      SqlStmt< 5> = 'from'
      SqlStmt< 6> = '   RT_LOG':JobNo
      SqlStmt< 7> = 'where'
      SqlStmt< 8> = '   EVAL "@ID" NOT LIKE ':"'//%'"
Mamu Kim
Post Reply