I am facing the following problem in one of the parallel jobs that I run using more than 1 node:
(The job contains only Input File sets, Output Seq files, Join Stages and Basic Transformer stages)
1) When the input to a job is changed after successfully running it several times with the previous input, the output for the run with the new input contains some data of the previous input !! (A new output file gets created every time the job is run as the stage is in overwrite mode)
2) As these jobs include a basic transformer, I tried increasing the value of environment variable DSIPC_OPEN_TIMEOUT. There was no change in the behaviour.
Here's what happens: Basic Transformer ---> Output Seq file. The monitor in director shows 68 output records from Transformer but the i/p to seq file shows 125 records.
Did u face something similar ?
SOS
Ravi Mohan
The above problem seems to be a memory related one. Any solutions/suggestions ?
PX Jobs on N Nodes: Inconsistent Output
Moderators: chulett, rschirm, roy
-
- Charter Member
- Posts: 88
- Joined: Tue Jan 13, 2004 3:07 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The row count discrepancy might be accounted for if there were internal end-of-line characters in the rows output from the Transfomer stage. These rows would, properly, be seen as multiple lines in a text file.
Just a thought.
Just a thought.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.