Classifying Link Types as Source, Reference and Reject

[muralisankarr]

Is there any way we have to find a link type as Stream, Reference or Reject from DS routines? I know there is a function to classify the stages DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE) but don't find any thing as such for links. I had a Routine called after every job in sequence which should do this piece of work. Please suggest

Many Thanks
MSR

[ArndW]

The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.

[muralisankarr]

The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.

Thanks Arnd for your reply. I got couple of issues in this regard

1) When I get the LINKTYPES for Dataset (as Target) and Lookup Fileset (as Source) then I'm getting Fatal error as given below

TEST #1
*******

Arg1 = 1

Test completed.

DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : DT025_CO08 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : CleanCO_08 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01D has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTime has the following links :- 1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Xfm has the following links :- 1,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : RejectTimeFiltredRecords has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E04 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTText has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01B has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E05 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferItem has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PrepareT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Aggregator has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : MapT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ConvertCH has the following links :- 1,3,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : SaveORACLE has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01File has the following links :- 1
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name PivotedE01E in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PivotedE01E has the following links :- -7
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name T01 in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01 has the following links :- -7

Result = 0

2) Is there any way we have to differentiate the Input and Reference link. The Linkype is 1 in both of the case. And again between Output and Reject Link. They both are represented as 3

Many Thanks
MSR

[ray.wurlod]

Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.

Without seeing your job design it's difficult to comment further. What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call?

[muralisankarr]

Thanks for you reply

Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.

Without seeing your job design it's difficult to comment further.
?

It is happening in all the jobs where we use Dataset (as source or target) and lookup fileset (as target)

What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call

I was using "JobHandle = DSAttachJob("DT025_LOAD_TTEXT_JBS", DSJ.ERRFATAL)" to attach the job

[ray.wurlod]

And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)

[muralisankarr]

And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)

My DSAttachJob was not creating the issue. If we see the original log pasted before we can see that the other stage types in the same job was displaying the link type. It is only the Dataset and Lookup fileset are creating the issue

I have attached my test routine here...

Code: Select all

*Attatch the Datastage API header with the routine
$IFNDEF JOBCONTROL
$INCLUDE DSINCLUDE JOBCONTROL.H
$ENDIF
*Initialize the Err code and Routine name
      ErrorCode = 0
      Equate RoutineName To 'ExtStat'
      ArrVar = ''
      FileHeader = "JobStartTime~JobEndTime~JobElapsedTime~StageType~StageName~LinkName~LnkRowCnt"
      Dir = "E:\TEMP"
      Suffix = ".Txt"
*Find the job & invocations and attach the handle to it
      JobHandle = DSAttachJob("B737_376_DT025_C_MAINT_GRP2_CMT_T06_P4_V1", DSJ.ERRFATAL)
      JobName = DSGetJobInfo(JobHandle,DSJ.JOBNAME)
      JobInvId = DSJobInvocationId
      If JobInvId ne "" Then
         JobName:= ".":JobInvId
      End
*Prepare the log file
      FilePath = Dir:JobName:Suffix
      Command = "TYPE nul >" : FilePath
      Call DSExecute ("NT", Command, Output, SystemReturnCode)
      If SystemReturnCode <> 0 Then
         Call DSLogFatal ("The file path for reconciliation is wrong" , RoutineName)
      End
*Get the Job Start, End and Elapsed timing
      JobStartTime = DSGetJobInfo ( JobHandle, DSJ.JOBSTARTTIMESTAMP )
      JobEndTime = DSGetJobInfo ( JobHandle, DSJ.JOBLASTTIMESTAMP )
      JobElapsedTime = DSGetJobInfo ( JobHandle, DSJ.JOBELAPSED)
*Get the list of Active and Passive stages
*      ActiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST)
*      PassiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST2 )
*      If ActiveStageLst ne "" And PassiveStageLst ne "" Then
*         StageLst = ActiveStageLst : "," : PassiveStageLst
*      End
*         Else If ActiveStageLst eq "" Then
*         StageLst = PassiveStageLst
*      End
*         Else If PassiveStageLst eq "" Then
*         StageLst = ActiveStageLst
*      End
      StageLst =  DSGetJobInfo ( JobHandle , DSJ.FULLSTAGELIST)
      StageCnt = DCount (StageLst, "," )
*Open the file and append the header
      OpenSeq FilePath to RFileVar Then
         WriteSeqF FileHeader to RFileVar
            Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
*Find the Stage and Link for which we need to take the Count
         For CtrStg = 1 to StageCnt
            StageName = Field (StageLst, ",",CtrStg)
            StageType = DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE)
            LinkTypeLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKTYPES)
            Call DSLogInfo ("The Stage : " : StageName : " has the following links :- " : LinkTypeLst , RoutineName)
            IsAuditStage = "_"
*Right (StageName,1)
            If IsAuditStage eq "_" Then
               LinkLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKLIST)
               LinkCnt = DCount (LinkLst,",")
               For CtrLnk = 1 to LinkCnt
                  LinkName = Field (LinkLst, ",", CtrLnk)
                  IsAuditLink = "_"
*Right (LinkName,1)
                  If IsAuditLink eq "_" Then
                     LnkRowCnt = DSGetLinkInfo (JobHandle, StageName, LinkName,DSJ.LINKROWCOUNT)
*                     Call DSLogInfo(" The Stage Name was " : StageName : "& The Link Name is " : LinkName : "& The Row count is " : LnkRowCnt, RoutineName)
                     FileLine = JobStartTime:"~":JobEndTime:"~":JobElapsedTime:"~":StageType:"~":StageName:"~":LinkName:"~":LnkRowCnt
                     WriteSeqF FileLine to RFileVar
                        Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
                  End
               Next LinkCnt
            End
         Next CtrStg
         CloseSeq RFileVar
      End
      Else
         Call DSLogFatal ("The reconciliation file was not is a correct stage " , RoutineName)
      End
      Ans = 0

[ArndW]

The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.

I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?

[muralisankarr]

The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.

I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?

Many Thanks for the reply. The pivot in the log was logical name given to a lookup file set. We will do a vertical pivot operation in a pre processing job and refer it in one of the main jon. And I have tested this function against all my jobs. Whenever we got dataset or lookup fileset then the error is logged!

[cassi_bsb]

muralisankarr,

Did you resolve it? I have same problem with DSGetStageInfo.

Cassiano

[chulett]

And what 'same problem' would that be, exactly?