Page 1 of 1
Classifying Link Types as Source, Reference and Reject
Posted: Tue Mar 09, 2010 6:14 am
by muralisankarr
Is there any way we have to find a link type as Stream, Reference or Reject from DS routines? I know there is a function to classify the stages DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE) but don't find any thing as such for links. I had a Routine called after every job in sequence which should do this piece of work. Please suggest
Many Thanks
MSR
Posted: Tue Mar 09, 2010 6:42 am
by ArndW
The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.
Posted: Tue Mar 09, 2010 10:22 pm
by muralisankarr
ArndW wrote:The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.
Thanks Arnd for your reply. I got couple of issues in this regard
1) When I get the LINKTYPES for Dataset (as Target) and Lookup Fileset (as Source) then I'm getting Fatal error as given below
TEST #1
*******
Arg1 = 1
Test completed.
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : DT025_CO08 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : CleanCO_08 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01D has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTime has the following links :- 1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Xfm has the following links :- 1,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : RejectTimeFiltredRecords has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E04 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTText has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01B has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E05 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferItem has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PrepareT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Aggregator has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : MapT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ConvertCH has the following links :- 1,3,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : SaveORACLE has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01File has the following links :- 1
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name PivotedE01E in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PivotedE01E has the following links :- -7
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name T01 in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01 has the following links :- -7
Result = 0
2) Is there any way we have to differentiate the Input and Reference link. The Linkype is 1 in both of the case. And again between Output and Reject Link. They both are represented as 3
Many Thanks
MSR
Posted: Tue Mar 09, 2010 10:25 pm
by ray.wurlod
Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.
Without seeing your job design it's difficult to comment further. What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call?
Posted: Tue Mar 09, 2010 10:49 pm
by muralisankarr
Thanks for you reply
ray.wurlod wrote:Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.
Without seeing your job design it's difficult to comment further.
?
It is happening in all the jobs where we use Dataset (as source or target) and lookup fileset (as target)
ray.wurlod wrote:
What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call
I was using "JobHandle = DSAttachJob("DT025_LOAD_TTEXT_JBS", DSJ.ERRFATAL)" to attach the job
Posted: Wed Mar 10, 2010 2:38 am
by ray.wurlod
And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)
Posted: Wed Mar 10, 2010 3:06 am
by muralisankarr
ray.wurlod wrote:And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)
My DSAttachJob was not creating the issue. If we see the original log pasted before we can see that the other stage types in the same job was displaying the link type. It is only the Dataset and Lookup fileset are creating the issue
I have attached my test routine here...
Code: Select all
*Attatch the Datastage API header with the routine
$IFNDEF JOBCONTROL
$INCLUDE DSINCLUDE JOBCONTROL.H
$ENDIF
*Initialize the Err code and Routine name
ErrorCode = 0
Equate RoutineName To 'ExtStat'
ArrVar = ''
FileHeader = "JobStartTime~JobEndTime~JobElapsedTime~StageType~StageName~LinkName~LnkRowCnt"
Dir = "E:\TEMP"
Suffix = ".Txt"
*Find the job & invocations and attach the handle to it
JobHandle = DSAttachJob("B737_376_DT025_C_MAINT_GRP2_CMT_T06_P4_V1", DSJ.ERRFATAL)
JobName = DSGetJobInfo(JobHandle,DSJ.JOBNAME)
JobInvId = DSJobInvocationId
If JobInvId ne "" Then
JobName:= ".":JobInvId
End
*Prepare the log file
FilePath = Dir:JobName:Suffix
Command = "TYPE nul >" : FilePath
Call DSExecute ("NT", Command, Output, SystemReturnCode)
If SystemReturnCode <> 0 Then
Call DSLogFatal ("The file path for reconciliation is wrong" , RoutineName)
End
*Get the Job Start, End and Elapsed timing
JobStartTime = DSGetJobInfo ( JobHandle, DSJ.JOBSTARTTIMESTAMP )
JobEndTime = DSGetJobInfo ( JobHandle, DSJ.JOBLASTTIMESTAMP )
JobElapsedTime = DSGetJobInfo ( JobHandle, DSJ.JOBELAPSED)
*Get the list of Active and Passive stages
* ActiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST)
* PassiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST2 )
* If ActiveStageLst ne "" And PassiveStageLst ne "" Then
* StageLst = ActiveStageLst : "," : PassiveStageLst
* End
* Else If ActiveStageLst eq "" Then
* StageLst = PassiveStageLst
* End
* Else If PassiveStageLst eq "" Then
* StageLst = ActiveStageLst
* End
StageLst = DSGetJobInfo ( JobHandle , DSJ.FULLSTAGELIST)
StageCnt = DCount (StageLst, "," )
*Open the file and append the header
OpenSeq FilePath to RFileVar Then
WriteSeqF FileHeader to RFileVar
Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
*Find the Stage and Link for which we need to take the Count
For CtrStg = 1 to StageCnt
StageName = Field (StageLst, ",",CtrStg)
StageType = DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE)
LinkTypeLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKTYPES)
Call DSLogInfo ("The Stage : " : StageName : " has the following links :- " : LinkTypeLst , RoutineName)
IsAuditStage = "_"
*Right (StageName,1)
If IsAuditStage eq "_" Then
LinkLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKLIST)
LinkCnt = DCount (LinkLst,",")
For CtrLnk = 1 to LinkCnt
LinkName = Field (LinkLst, ",", CtrLnk)
IsAuditLink = "_"
*Right (LinkName,1)
If IsAuditLink eq "_" Then
LnkRowCnt = DSGetLinkInfo (JobHandle, StageName, LinkName,DSJ.LINKROWCOUNT)
* Call DSLogInfo(" The Stage Name was " : StageName : "& The Link Name is " : LinkName : "& The Row count is " : LnkRowCnt, RoutineName)
FileLine = JobStartTime:"~":JobEndTime:"~":JobElapsedTime:"~":StageType:"~":StageName:"~":LinkName:"~":LnkRowCnt
WriteSeqF FileLine to RFileVar
Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
End
Next LinkCnt
End
Next CtrStg
CloseSeq RFileVar
End
Else
Call DSLogFatal ("The reconciliation file was not is a correct stage " , RoutineName)
End
Ans = 0
Posted: Wed Mar 10, 2010 3:17 am
by ArndW
The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.
I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?
Posted: Wed Mar 10, 2010 4:12 am
by muralisankarr
ArndW wrote:The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.
I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?
Many Thanks for the reply. The pivot in the log was logical name given to a lookup file set. We will do a vertical pivot operation in a pre processing job and refer it in one of the main jon. And I have tested this function against all my jobs. Whenever we got dataset or lookup fileset then the error is logged!
Posted: Thu Jul 15, 2010 1:30 pm
by cassi_bsb
muralisankarr,
Did you resolve it? I have same problem with DSGetStageInfo.
Cassiano
Posted: Thu Jul 15, 2010 2:09 pm
by chulett
And what 'same problem' would that be, exactly?