Wrapper Stage

[pravin1581]

Hi All,

I have used a new parallel stage- wrapper that calls a C program , takes its input from the sequential file stage preceding it and produces the output in another file. This stage runs fine when in a stand alone job. But when i integrate it in another job with the output from a sort stage as an input to the wrapper stage then it is not working . The job performance monitor shows that from sort stage X no. of rows are entering and 0 rows are getting out of the wrapper stage . i.e. the wrapper stage is not working and the output file has no record. Can anyone provide a solution , may be wrapper stage expects input from a files stage only.

[DSguru2B]

Is there any error message in the log?

[rameshrr3]

I have used wrapped stages with Non -File inputs. Dont think that should be the issue.

Thanks
Ramesh

[pravin1581]

Is there any error message in the log?

There is no error message in the log. It is only showing export complete .0 rows exported successfully.

[pravin1581]

I have used wrapped stages with Non -File inputs. Dont think that should be the issue.

Thanks
Ramesh

In my case the wrapper stage has its input from the sort stage.

[nick.bond]

Only an idea but try adding the $APT_DISABLE_COMBINATION environment variable to the job, setting it to True.

[pravin1581]

Only an idea but try adding the $APT_DISABLE_COMBINATION environment variable to the job, setting it to True.

How will the $APT_DISABLE_COMBINATION variable affect the wrapper stage , I was going thru the manuals and they are clearly stating that not to change the $APT_DISABLE_COMBINATION variable as it may affect the performance of the job.

[nick.bond]

EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.

I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?

I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.

[pravin1581]

EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.

I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?

I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.

Ok I will use it in my job.

[pravin1581]

EE will combine oporators in the job to improve performance, I can't see how it would try and combine a sort state and your rapper stage, but if you can see the data going into the sort stage and not going from your sort to your rapper stage i would just add $APT_DISABLE_COMBINATION = True as a means to help debug.

I don't really expect this to fix the problme but whenever somthing is happening in and EE job that can't be explained it is one option to help debug. Why not just give it a go?

I have had a job that had to have this set otherwise it wouldn't run on a single node. It is not something that "should never be used", you should use it when it helps.

Ok I will use it in my job.

The property has been set to TRUE , but the job log is hard to debug.

[ray.wurlod]

Read each warning carefully - the name of the stage and explanatory text is given, often with other diagnostic information. Why not post one of the warnings, together with your understanding of what it is telling you? If you have version 8.0 there is also an error codes reference manual, though this is incomplete.

[pravin1581]

Read each warning carefully - the name of the stage and explanatory text is given, often with other diagnostic information. Why not post one of the warnings, together with your understanding of what it is telling you? If you have version 8.0 there is also an error codes reference manual, though this is incomplete.

Here is the entry from the job log where Parallel Job Initiated information is given.....

Code: Select all

    
         STAGE: Median_19
## Operator
Median.op
## General options
[ident('Median_19'); jobmon_ident('Median_19')]
## Inputs
0< [] 'sort_UPC_Org:ln_ip_med_file.v'
## Outputs
0> [modify (
  Org:ustring[max=255]=Org;
)] 'Median_19:ln_op_Median.v'
;

 

The Stage : Median_19 is the wrapper stage used in the job. It takes its input from the SORT stage and produces the output in the sequential file.

[ray.wurlod]

No it doesn't - it produces its output into the virtual Data Set whose control file name is 'Median_19:ln_op_Median.v'

What you have posted is a small piece of the generated OSH. While it is true that this is logged, it is NOT the event that reports any error.

Is there any event in the log with a severity of Warning (yellow icon) or Error (red icon)? If so, please post the detail of these events. If not, your job is working perfectly (or you've disabled warnings, which is always done at your own peril).