DSGetStageInfo in Parallel job
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
DSGetStageInfo in Parallel job
Hi All,
I have requirement where I need to derive the data corresponding to the row number. Same thing was achieved in server job using DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) in the constraint of the transformer stage but as DSGetStageInfo is not there in the parallel job, how can the same be done.
I have requirement where I need to derive the data corresponding to the row number. Same thing was achieved in server job using DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) in the constraint of the transformer stage but as DSGetStageInfo is not there in the parallel job, how can the same be done.
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
Re: DSGetStageInfo in Parallel job
pravin1581 wrote:Hi All,
I have requirement where I need to derive the data corresponding to the row number. Same thing was achieved in server job using DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) in the constraint of the transformer stage but as DSGetStageInfo is not there in the parallel job, how can the same be done.
the same function is available in parellel also
syntax:
Result = DSGetStageInfo (JobHandle, StageName, InfoType)
JobHandle is the handle for the job as derived from DSAttachJob, or it
may be DSJ.ME to refer to the current job.
StageName is the name of the stage to be interrogated. It may also be
DSJ.ME to refer to the current stage.
InfoType specifies the information required it may be DSJ.STAGEINROWNUM
try to use it in routine
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.
Genius may have its limitations, but stupidity is not thus handicapped.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Re: DSGetStageInfo in Parallel job
can i directly use in the transformer constraint or i need to write a routine for it and then call that routine in the transformer.priyadarshikunal wrote:pravin1581 wrote:Hi All,
I have requirement where I need to derive the data corresponding to the row number. Same thing was achieved in server job using DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) in the constraint of the transformer stage but as DSGetStageInfo is not there in the parallel job, how can the same be done.
the same function is available in parellel also
syntax:
Result = DSGetStageInfo (JobHandle, StageName, InfoType)
JobHandle is the handle for the job as derived from DSAttachJob, or it
may be DSJ.ME to refer to the current job.
StageName is the name of the stage to be interrogated. It may also be
DSJ.ME to refer to the current stage.
InfoType specifies the information required it may be DSJ.STAGEINROWNUM
try to use it in routine
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
Re: DSGetStageInfo in Parallel job
pravin1581 wrote:can i directly use in the transformer constraint or i need to write a routine for it and then call that routine in the transformer.priyadarshikunal wrote:pravin1581 wrote:Hi All,
I have requirement where I need to derive the data corresponding to the row number. Same thing was achieved in server job using DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) in the constraint of the transformer stage but as DSGetStageInfo is not there in the parallel job, how can the same be done.
the same function is available in parellel also
syntax:
Result = DSGetStageInfo (JobHandle, StageName, InfoType)
JobHandle is the handle for the job as derived from DSAttachJob, or it
may be DSJ.ME to refer to the current job.
StageName is the name of the stage to be interrogated. It may also be
DSJ.ME to refer to the current stage.
InfoType specifies the information required it may be DSJ.STAGEINROWNUM
try to use it in routine
the above function returns a value depending on InfoType provided no matter where u use it, after that it depends on u how to handle the result.
ur query will return Primary links input row number.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.
Genius may have its limitations, but stupidity is not thus handicapped.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
How can a running total be of any help in this case , i need the values corresponding to the row number.ray.wurlod wrote:You won't get that from DSGetStageInfo().
Use a stage variable to keep the running total.
rownum ID
---------------------------
10 5
11 4
I need the values 5 and 4 in the output file.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
How did you propose to get it from DSGetStageInfo()?!!
Is the source a Sequential File? If so you can use the Row Number Column property to yield the row number from the file - even if you are reading multiple files or using multiple readers per node.
Presumably - because you've never said - the value of ID comes from the row you've imported.
Is the source a Sequential File? If so you can use the Row Number Column property to yield the row number from the file - even if you are reading multiple files or using multiple readers per node.
Presumably - because you've never said - the value of ID comes from the row you've imported.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
The source is the sequential file and DSGetStageInfo(DSJ.ME, DSJ.ME, DSJ.STAGEINROWNUM) is working in the server job to return the value corresponding to the row.ray.wurlod wrote:How did you propose to get it from DSGetStageInfo()?!!
Is the source a Sequential File? If so you can use the Row Number Column property to yield the row number from the file - even if you are reading multiple files or using multiple readers per node.
Presumably - because you've never said - the value of ID comes from the row you've imported.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
So what?
BASIC functions do not work in the C++ environment of parallel jobs.
If you want to write your own C++ equivalent of DSGetStageInfo() as a parallel routine - and you prefer such an inefficient approach - then there is a C-callable DataStage API that offers a DSGetStageInfo() function.
BASIC functions do not work in the C++ environment of parallel jobs.
If you want to write your own C++ equivalent of DSGetStageInfo() as a parallel routine - and you prefer such an inefficient approach - then there is a C-callable DataStage API that offers a DSGetStageInfo() function.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
C-callable DataStage API , can u please elaborate on that part as I am new to parallel jobs and we will be needing quite a few number of DS Functions in our job as we have done in the server job. Please elaborate that how can DS Functions be used in the PX job , they are not available in the transformer.ray.wurlod wrote:So what?
BASIC functions do not work in the C++ environment of parallel jobs.
If you want to write your own C++ equivalent of DSGetStageInfo() as a parallel routine - and you prefer such an inefficient approach - then there is a C-callable DataStage API that offers a DSGetStageInfo() function.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
DS functions can not be used in the parallel job until they are available from the Transformer (expression editor).
For this to happen, you have to create the parallel routines.
A parallel routine in the Repository is merely an interlude to a C++ function that you yourself have written, compiled and linked. And, presumably, tested.
Within that routine, you can call functions in the DataStage API. The functions in the API are documented in Chapter 7 of the Parallel Job Advanced Developer's Guide
There is nothing "straight out of the box", which is what you appear to be expecting.
For this to happen, you have to create the parallel routines.
A parallel routine in the Repository is merely an interlude to a C++ function that you yourself have written, compiled and linked. And, presumably, tested.
Within that routine, you can call functions in the DataStage API. The functions in the API are documented in Chapter 7 of the Parallel Job Advanced Developer's Guide
There is nothing "straight out of the box", which is what you appear to be expecting.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
But when we create some routines it asks for the library path to save it.ray.wurlod wrote:DS functions can not be used in the parallel job until they are available from the Transformer (expression editor).
For this to happen, you have to create the parallel routines.
A parallel routine in the Repository is merely an interlude to a C++ function that you yourself have written, compiled and linked. And, presumably, tested.
Within that routine, you can call functions in the DataStage API. The functions in the API are documented in Chapter 7 of the Parallel Job Advanced Developer's Guide
There is nothing "straight out of the box", which is what you appear to be expecting.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If that requirement bewilders you, hire a competent C++ programmer.
What you are creating in the Repository is not the routine itself, but an interlude that allows the C++ function to be found.
It is not asking you for a library in which to save anything - it is asking you for the pathname of the library in which the C++ function can be found.
What you are creating in the Repository is not the routine itself, but an interlude that allows the C++ function to be found.
It is not asking you for a library in which to save anything - it is asking you for the pathname of the library in which the C++ function can be found.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.