DSXchange

ray.wurlod wrote:DataStage job logs are located on the server.

However they are not log files, as you have assumed. They are database tables in the repository.

If you determine the job number of your job, from the DS_JOBS table, then the log for that job is a table called RT_LOGnnn, where nnn is the job number.

You can view them from the operating system using the command line interface dsjob with its -log... options.

You can view them from the Administrator client command window with a regular query, for example

Code: Select all

SELECT * FROM RT_LOGnnn;

Search the forum for more information about viewing and archiving DataStage job logs.

rajeev_prabhuat wrote:Hi,

Ray Thanks you.

But when i was going through the DS documents that we get during instation, i came to see that log files are stored in hash files. Is there anyway to reduce the size of the log files?.

Regards,
Rajeev Prabhu

ray.wurlod wrote:DataStage job logs are located on the server.

However they are not log files, as you have assumed. They are database tables in the repository.

If you determine the job number of your job, from the DS_JOBS table, then the log for that job is a table called RT_LOGnnn, where nnn is the job number.

You can view them from the operating system using the command line interface dsjob with its -log... options.

You can view them from the Administrator client command window with a regular query, for example

Code: Select all

SELECT * FROM RT_LOGnnn;

Search the forum for more information about viewing and archiving DataStage job logs.

ray.wurlod wrote:Hashed file is a method of implementing database tables in DataStage, UniVerse, UniData and other databases.

Columns in the log files are:

Code: Select all

@ID          Numeric event identifier (or "//..." for control records)
TYPE         Enumerated severity level of message
WAVE.NO      Internal number to identify run instance
TIMESTAMP    Date and time when event logged
MSG.ARGS     Parameters to add to message text (multi-valued)
MSG.TEXT     Fixed part of message text
FULL.TEXT    MSG.TEXT with parameters substituted from MSG.ARGS
SEVERITY     TYPE decoded into a text string

The first three are INTEGER, the remainder are VARCHAR.

• DSGetLogSummary
  DSGetNewestLogId
  DSGetLogEntry

ray.wurlod wrote:You can create a DataStage job that retrieves the log information using the metadata I supplied to retrieve using either a UV stage or a hashed file stage. If you use a hashed file stage, the key column should be called AT.ID rather than @ID. Either stage type can constraint on any column, in the usual way.

From there you can do whatever you like with the contents of the log file.

If you feel up to the challenge, you can learn about the log interrogation functions from on-line help. They also have equivalents in the dsjob command line options.

• DSGetLogSummary
  DSGetNewestLogId
  DSGetLogEntry

If you are getting large log files, remember to purge them regularly. If a log reaches 2GB, and preventive action (switching to 64-bit internal addressing) has not been taken, the log file becomes corrupted and your job is guaranteed to abort.

ray.wurlod wrote:You can create a DataStage job that retrieves the log information using the metadata I supplied to retrieve using either a UV stage or a hashed file stage. If you use a hashed file stage, the key column should be called AT.ID rather than @ID. Either stage type can constraint on any column, in the usual way.

From there you can do whatever you like with the contents of the log file.

If you feel up to the challenge, you can learn about the log interrogation functions from on-line help. They also have equivalents in the dsjob command line options.

• DSGetLogSummary
  DSGetNewestLogId
  DSGetLogEntry

If you are getting large log files, remember to purge them regularly. If a log reaches 2GB, and preventive action (switching to 64-bit internal addressing) has not been taken, the log file becomes corrupted and your job is guaranteed to abort.

ray.wurlod wrote:The fundamental requirement is to get the table name correct!

SELECT TIMESTAMP, SEVERITY, FULL.TEXT FROM RT_LOG26 ;

  $IFNDEF JOBCONTROL.H 
$INCLUDE DSINCLUDE JOBCONTROL.H 
$ENDIF 

ErrorCode = 0;* set this to non-zero to stop the stage/job

ReportType = 0
LogEntries = 0
DirName = InputArg
Suffix = ".txt"
JobHandle = DSJ.ME
JobName = DSGetJobInfo(JobHandle,DSJ.JOBNAME)
JobInvId = DSJobInvocationId
if JobInvId ne "" then
   JobName:= ".":JobInvId
end

JobStarted = convert(" :-","_",DSGetJobInfo(JobHandle,DSJ.JOBSTARTTIMESTAMP))
JobEnded = convert(" :-","_",DSGetJobInfo(JobHandle,DSJ.JOBEOTTIMESTAMP))

Alias = ""
if JobInvId eq "" then
  Alias = DSGetIdForJob(JobName,ErrorCode)
end

if Alias ne "" then
  FileName = JobName:"_":Alias
end
else
  FileName = JobName:"_":JobStarted
end

FileName := Suffix

ReportText = DSMakeJobReport(JobHandle,ReportType,"LF")
*- Getting the required counts from the Job Links
varSrcCount1 = DSGetLinkInfo(JobHandle,'Sequential_File_4','In',DSJ.LINKROWCOUNT)

LogEntries[10] = DSGetLogSummary(JobHandle,DSJ.LOGINFO,JobStarted,JobEnded,0)

varSrcCount2 = 0
varInsCount=0
varSrcCount = varSrcCount1 + varSrcCount2
varUpdCount = 0
varDelCount = 0
Audit_info='"1",'
Audit_info := '"':Field(trims(Field(ReportText,CHAR(10),3)),' ',5):'",'
Audit_info :='"':'LOADING':'",'
Audit_info := '"':right(trims(Field(ReportText,CHAR(10),5)),19):'",'
Audit_info := '"':right(trims(Field(ReportText,CHAR(10),6)),19):'",'
Audit_info := '"':right(trims(Field(ReportText,CHAR(10),7)),8):'",'
Audit_info := '"':right(Field(trims(Field(ReportText,CHAR(10),8)),' ',2),0):'",'
Audit_info := varSrcCount:','
Audit_info := varInsCount:','
Audit_info := varUpdCount:','
Audit_info := varDelCount:','
Audit_info := LogEntries[4]:'"'

Openpath DirName to t.fvar then
  write Audit_info to t.fvar, FileName else
  call DSLogWarn("Failed to write file ":FileName:" to directory ":DirName, "Routine_call_check")
end
end
else
call DSLogWarn("Failed to open directory1 ":DirName, "Routine_call_check")
end
close t.fvar
 
*- Audit Code
AuditText =DSMakeJobReport(JobHandle,1,"LF")
AuditFileName := "Audit_":FileName
Openpath DirName to t.fvar then
write AuditText to t.fvar, AuditFileName else
call DSLogWarn("Failed to write file ":AuditFileName:" to directory ":DirName, "Routine_call_check")
end
end
else
call DSLogWarn("Failed to open directory ":DirName, "Routine_call_check")
end
close t.fvar

ray.wurlod wrote:Need to see your whole routine. Post it here, surrounded by Code tags (select the whole code, then click on the Code button above).

The JobHandle variable must have been assigned by an earlier DSAttachJob() call.
The result from DSGetLogSummary is a "dynamic array". Fields in dynamic arrays are referred to using angle bracket notation.

Result = DSGetLogSummary(...)
FirstField = Result<1>
SecondField = Result<2>

and so on.

You can also use the MATPARSE statement (refer to DataStage BASIC manual) to load the contents of a dynamic array into a dimensioned array. Dimensioned array elements are accessed with the index value in parentheses (not square brackets).