Sequential operator cannot preserve partitioning of parallel
Moderators: chulett, rschirm, roy
Sequential operator cannot preserve partitioning of parallel
Hi,
I am getting a warning message while validating my job. I am writing the trailer records into a sequential file using tail stage and i am getting this datastage warning:
Sequential_File_12: When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0
Please anybody throw some light on this!!!
Thanks in advance!!!
Srikanth Reddy
I am getting a warning message while validating my job. I am writing the trailer records into a sequential file using tail stage and i am getting this datastage warning:
Sequential_File_12: When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0
Please anybody throw some light on this!!!
Thanks in advance!!!
Srikanth Reddy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Anything that requires sequential file I/O is limited by the operating system to a single writing process. (This is not a DataStage limitation.) The warning is issued to alert you to the fact that your design may introduce a processing bottleneck.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 25
- Joined: Thu Oct 02, 2003 8:57 am
Also, this warning is caused by a preceeding stage having the "Preserve Partitioning" flag set to "Propagate". This is the default value, and tells PX that it should attempt to maintain a partitioning method. Obviously the sequential stage can't preserve the partitioning. If this option is set to "Clear", the sequential stage wont report a warning (i dont like to see warnings if i can stop them :D )
Hope this helps a bit.
Hope this helps a bit.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I agree. Particularly in a production environment you should ensure that there are no warnings.
Then, when there are warnings, they are truly indicative that a problem has occurred that needs investigation. By this means I once discovered that "they" had changed the format of the source file without telling anyone (14 characters in one field rather than 12 - suddenly the job threw a data truncation warning).
Then, when there are warnings, they are truly indicative that a problem has occurred that needs investigation. By this means I once discovered that "they" had changed the format of the source file without telling anyone (14 characters in one field rather than 12 - suddenly the job threw a data truncation warning).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 48
- Joined: Thu Mar 11, 2004 10:32 pm
Partitioning is NOT eliminated.
It is just a matter of:
"If you need to repartition this data, feel free."
instead of:
"You can not repartition this data."
Sequential stages must repartition, so that's why you got the warning when you told it (by default) to not repartition, and you got ignored. (How rude!)
It is still being done in parallel.
It is just a matter of:
"If you need to repartition this data, feel free."
instead of:
"You can not repartition this data."
Sequential stages must repartition, so that's why you got the warning when you told it (by default) to not repartition, and you got ignored. (How rude!)
It is still being done in parallel.
The problem is some stages don't have this option, such as DB2 Connector to reject link. Without this option to clear, how do you clear this warning?The warning is so obvious, why it produces this warning? It looks so dummy.leo_t_nice wrote:Also, this warning is caused by a preceeding stage having the "Preserve Partitioning" flag set to "Propagate". This is the default value, and tells PX that it should attempt to maintain a partitioning method. Obviously the sequential stage can't preserve the partitioning. If this option is set to "Clear", the sequential stage wont report a warning (i dont like to see warnings if i can stop them :D )
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: