Wrapper Stage
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Wrapper Stage
Hi All,
I have used a new parallel stage- wrapper that calls a C program , takes its input from the sequential file stage preceding it and produces the output in another file. This stage runs fine when in a stand alone job. But when i integrate it in another job with the output from a sort stage as an input to the wrapper stage then it is not working . The job performance monitor shows that from sort stage X no. of rows are entering and 0 rows are getting out of the wrapper stage . i.e. the wrapper stage is not working and the output file has no record. Can anyone provide a solution , may be wrapper stage expects input from a files stage only.
I have used a new parallel stage- wrapper that calls a C program , takes its input from the sequential file stage preceding it and produces the output in another file. This stage runs fine when in a stand alone job. But when i integrate it in another job with the output from a sort stage as an input to the wrapper stage then it is not working . The job performance monitor shows that from sort stage X no. of rows are entering and 0 rows are getting out of the wrapper stage . i.e. the wrapper stage is not working and the output file has no record. Can anyone provide a solution , may be wrapper stage expects input from a files stage only.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
How will the $APT_DISABLE_COMBINATION variable affect the wrapper stage , I was going thru the manuals and they are clearly stating that not to change the $APT_DISABLE_COMBINATION variable as it may affect the performance of the job.nick.bond wrote:Only an idea but try adding the $APT_DISABLE_COMBINATION environment variable to the job, setting it to True.
EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.
I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?
I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.
I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?
I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.
Regards,
Nick.
Nick.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Ok I will use it in my job.nick.bond wrote:EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.
I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?
I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
The property has been set to TRUE , but the job log is hard to debug.pravin1581 wrote:Ok I will use it in my job.nick.bond wrote:EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.
I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?
I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Read each warning carefully - the name of the stage and explanatory text is given, often with other diagnostic information. Why not post one of the warnings, together with your understanding of what it is telling you? If you have version 8.0 there is also an error codes reference manual, though this is incomplete.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 497
- Joined: Sun Dec 17, 2006 11:52 pm
- Location: Kolkata
- Contact:
Here is the entry from the job log where Parallel Job Initiated information is given.....ray.wurlod wrote:Read each warning carefully - the name of the stage and explanatory text is given, often with other diagnostic information. Why not post one of the warnings, together with your understanding of what it is telling you? If you have version 8.0 there is also an error codes reference manual, though this is incomplete.
Code: Select all
STAGE: Median_19
## Operator
Median.op
## General options
[ident('Median_19'); jobmon_ident('Median_19')]
## Inputs
0< [] 'sort_UPC_Org:ln_ip_med_file.v'
## Outputs
0> [modify (
Org:ustring[max=255]=Org;
)] 'Median_19:ln_op_Median.v'
;
The Stage : Median_19 is the wrapper stage used in the job. It takes its input from the SORT stage and produces the output in the sequential file.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
No it doesn't - it produces its output into the virtual Data Set whose control file name is 'Median_19:ln_op_Median.v'
What you have posted is a small piece of the generated OSH. While it is true that this is logged, it is NOT the event that reports any error.
Is there any event in the log with a severity of Warning (yellow icon) or Error (red icon)? If so, please post the detail of these events. If not, your job is working perfectly (or you've disabled warnings, which is always done at your own peril).
What you have posted is a small piece of the generated OSH. While it is true that this is logged, it is NOT the event that reports any error.
Is there any event in the log with a severity of Warning (yellow icon) or Error (red icon)? If so, please post the detail of these events. If not, your job is working perfectly (or you've disabled warnings, which is always done at your own peril).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.