Classifying Link Types as Source, Reference and Reject
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 83
- Joined: Tue Oct 28, 2008 1:55 am
- Location: Chennai
Classifying Link Types as Source, Reference and Reject
Is there any way we have to find a link type as Stream, Reference or Reject from DS routines? I know there is a function to classify the stages DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE) but don't find any thing as such for links. I had a Routine called after every job in sequence which should do this piece of work. Please suggest
Many Thanks
MSR
Many Thanks
MSR
The minute you start talking about what you're going to do if you lose, you have lost
The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Premium Member
- Posts: 83
- Joined: Tue Oct 28, 2008 1:55 am
- Location: Chennai
Thanks Arnd for your reply. I got couple of issues in this regardArndW wrote:The DSGetStageInfo() using DSJ.LINKTYPES will get you that information.
1) When I get the LINKTYPES for Dataset (as Target) and Lookup Fileset (as Source) then I'm getting Fatal error as given below
2) Is there any way we have to differentiate the Input and Reference link. The Linkype is 1 in both of the case. And again between Output and Reject Link. They both are represented as 3TEST #1
*******
Arg1 = 1
Test completed.
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : DT025_CO08 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : CleanCO_08 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01D has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTime has the following links :- 1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Xfm has the following links :- 1,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : RejectTimeFiltredRecords has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E04 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferTText has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E01B has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : E05 has the following links :- 3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ReferItem has the following links :- 1,1,1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PrepareT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : Aggregator has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : MapT01 has the following links :- 1,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : ConvertCH has the following links :- 1,3,3,3
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : SaveORACLE has the following links :- 1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01File has the following links :- 1
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name PivotedE01E in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : PivotedE01E has the following links :- -7
DSLogFatal called from : DSGetStageInfo
Message to be logged is...
> Job control fatal error (-7)
> (DSGetStageInfo) Unknown stage name T01 in job B737_376_DT025_C_MAINT_GRP2_CMT_T01_P2_V1
DSLogInfo called from : ExtStat
Message to be logged is...
> The Stage : T01 has the following links :- -7
Result = 0
Many Thanks
MSR
The minute you start talking about what you're going to do if you lose, you have lost
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.
Without seeing your job design it's difficult to comment further. What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call?
Without seeing your job design it's difficult to comment further. What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 83
- Joined: Tue Oct 28, 2008 1:55 am
- Location: Chennai
Thanks for you reply
It is happening in all the jobs where we use Dataset (as source or target) and lookup fileset (as target)ray.wurlod wrote:Any negative number returned from a DataStage API function indicates an error. The error codes are documented in JOBCONTROL.H header file in DSINCLUDE. You can probably search DSXchange to find them.
Without seeing your job design it's difficult to comment further.
?
I was using "JobHandle = DSAttachJob("DT025_LOAD_TTEXT_JBS", DSJ.ERRFATAL)" to attach the jobray.wurlod wrote: What are you using for the job handle? DSJ.ME or the result of a DSAttachJob() call
The minute you start talking about what you're going to do if you lose, you have lost
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 83
- Joined: Tue Oct 28, 2008 1:55 am
- Location: Chennai
My DSAttachJob was not creating the issue. If we see the original log pasted before we can see that the other stage types in the same job was displaying the link type. It is only the Dataset and Lookup fileset are creating the issueray.wurlod wrote:And what was the result of DSAttachJob() ? (The best way to check is with some innocuous function like DSGetJobInfo(JobHandle, DSJ.JOBNAME) and log the value of the result. If this returns DSJE.BADHANDLE then your attach failed.)
I have attached my test routine here...
Code: Select all
*Attatch the Datastage API header with the routine
$IFNDEF JOBCONTROL
$INCLUDE DSINCLUDE JOBCONTROL.H
$ENDIF
*Initialize the Err code and Routine name
ErrorCode = 0
Equate RoutineName To 'ExtStat'
ArrVar = ''
FileHeader = "JobStartTime~JobEndTime~JobElapsedTime~StageType~StageName~LinkName~LnkRowCnt"
Dir = "E:\TEMP"
Suffix = ".Txt"
*Find the job & invocations and attach the handle to it
JobHandle = DSAttachJob("B737_376_DT025_C_MAINT_GRP2_CMT_T06_P4_V1", DSJ.ERRFATAL)
JobName = DSGetJobInfo(JobHandle,DSJ.JOBNAME)
JobInvId = DSJobInvocationId
If JobInvId ne "" Then
JobName:= ".":JobInvId
End
*Prepare the log file
FilePath = Dir:JobName:Suffix
Command = "TYPE nul >" : FilePath
Call DSExecute ("NT", Command, Output, SystemReturnCode)
If SystemReturnCode <> 0 Then
Call DSLogFatal ("The file path for reconciliation is wrong" , RoutineName)
End
*Get the Job Start, End and Elapsed timing
JobStartTime = DSGetJobInfo ( JobHandle, DSJ.JOBSTARTTIMESTAMP )
JobEndTime = DSGetJobInfo ( JobHandle, DSJ.JOBLASTTIMESTAMP )
JobElapsedTime = DSGetJobInfo ( JobHandle, DSJ.JOBELAPSED)
*Get the list of Active and Passive stages
* ActiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST)
* PassiveStageLst = DSGetJobInfo ( JobHandle , DSJ.STAGELIST2 )
* If ActiveStageLst ne "" And PassiveStageLst ne "" Then
* StageLst = ActiveStageLst : "," : PassiveStageLst
* End
* Else If ActiveStageLst eq "" Then
* StageLst = PassiveStageLst
* End
* Else If PassiveStageLst eq "" Then
* StageLst = ActiveStageLst
* End
StageLst = DSGetJobInfo ( JobHandle , DSJ.FULLSTAGELIST)
StageCnt = DCount (StageLst, "," )
*Open the file and append the header
OpenSeq FilePath to RFileVar Then
WriteSeqF FileHeader to RFileVar
Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
*Find the Stage and Link for which we need to take the Count
For CtrStg = 1 to StageCnt
StageName = Field (StageLst, ",",CtrStg)
StageType = DSGetStageInfo(JobHandle, StageName, DSJ.STAGETYPE)
LinkTypeLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKTYPES)
Call DSLogInfo ("The Stage : " : StageName : " has the following links :- " : LinkTypeLst , RoutineName)
IsAuditStage = "_"
*Right (StageName,1)
If IsAuditStage eq "_" Then
LinkLst = DSGetStageInfo(JobHandle, StageName,DSJ.LINKLIST)
LinkCnt = DCount (LinkLst,",")
For CtrLnk = 1 to LinkCnt
LinkName = Field (LinkLst, ",", CtrLnk)
IsAuditLink = "_"
*Right (LinkName,1)
If IsAuditLink eq "_" Then
LnkRowCnt = DSGetLinkInfo (JobHandle, StageName, LinkName,DSJ.LINKROWCOUNT)
* Call DSLogInfo(" The Stage Name was " : StageName : "& The Link Name is " : LinkName : "& The Row count is " : LnkRowCnt, RoutineName)
FileLine = JobStartTime:"~":JobEndTime:"~":JobElapsedTime:"~":StageType:"~":StageName:"~":LinkName:"~":LnkRowCnt
WriteSeqF FileLine to RFileVar
Else Call DSLogFatal ("Write opetation failed at reconciliation file" , RoutineName)
End
Next LinkCnt
End
Next CtrStg
CloseSeq RFileVar
End
Else
Call DSLogFatal ("The reconciliation file was not is a correct stage " , RoutineName)
End
Ans = 0
The minute you start talking about what you're going to do if you lose, you have lost
The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.
I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?
I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Premium Member
- Posts: 83
- Joined: Tue Oct 28, 2008 1:55 am
- Location: Chennai
Many Thanks for the reply. The pivot in the log was logical name given to a lookup file set. We will do a vertical pivot operation in a pre processing job and refer it in one of the main jon. And I have tested this function against all my jobs. Whenever we got dataset or lookup fileset then the error is logged!ArndW wrote:The '-' sign in your output line was confusing, it made it look like negative value lists were being returned.
I can't check now, but perhaps the pivot stage is incompatible. What happens if you write a small dummy job using the px pivot and seeing if you get the same abort?
The minute you start talking about what you're going to do if you lose, you have lost