Sequential operator cannot preserve partitioning of parallel

rsrikant · Post by **rsrikant** » Mon Apr 26, 2004 4:21 am

Hi,

I am getting a warning message while validating my job. I am writing the trailer records into a sequential file using tail stage and i am getting this datastage warning:

Sequential_File_12: When checking operator: Sequential operator cannot preserve partitioning of parallel dataset on input port 0

Please anybody throw some light on this!!!

Thanks in advance!!!
Srikanth Reddy

ray.wurlod · Post by **ray.wurlod** » Mon Apr 26, 2004 10:49 pm

Anything that requires sequential file I/O is limited by the operating system to a single writing process. (This is not a DataStage limitation.) The warning is issued to alert you to the fact that your design may introduce a processing bottleneck.

rsrikant · Post by **rsrikant** » Wed Apr 28, 2004 4:16 am

Thank you

leo_t_nice · Post by **leo_t_nice** » Thu Jun 03, 2004 11:18 am

Also, this warning is caused by a preceeding stage having the "Preserve Partitioning" flag set to "Propagate". This is the default value, and tells PX that it should attempt to maintain a partitioning method. Obviously the sequential stage can't preserve the partitioning. If this option is set to "Clear", the sequential stage wont report a warning (i dont like to see warnings if i can stop them :D )

Hope this helps a bit.

rsrikant · Post by **rsrikant** » Fri Jun 04, 2004 11:04 am

Hi Leo,

Thank you for the fix. No warnings are coming after changing the preceeding stage "Preserve Partitioning" flag to "Clear".

Me too don't like to see warnings as long as they can be avoided.

Thanks anyway.

Srikanth

ray.wurlod · Post by **ray.wurlod** » Fri Jun 04, 2004 5:30 pm

I agree. Particularly in a production environment you should ensure that there are no warnings.
Then, when there are warnings, they are truly indicative that a problem has occurred that needs investigation. By this means I once discovered that "they" had changed the format of the source file without telling anyone (14 characters in one field rather than 12 - suddenly the job threw a data truncation warning).

mandyli · Post by **mandyli** » Sun Jun 06, 2004 5:43 am

Hi

Any way changing Preserve Partitioning" flag to "Clear” it will not give any warning but during the production run u will check with same setup b'cus all the job running through Job Sequencer, the same job calling from jobsequencer check the warning.

varshanswamy · Post by **varshanswamy** » Tue May 03, 2005 1:23 pm

I would like to know if there are 2 outputs from a single input one output leading to a sequential file other to a dataset, then how is it possible to solve the problem, because the entire partitioning is eliminted

T42 · Post by **T42** » Mon May 09, 2005 8:53 pm

Partitioning is NOT eliminated.

It is just a matter of:

"If you need to repartition this data, feel free."

instead of:

"You can not repartition this data."

Sequential stages must repartition, so that's why you got the warning when you told it (by default) to not repartition, and you got ignored. (How rude!)

It is still being done in parallel.

olgc · Post by **olgc** » Mon Apr 13, 2015 9:51 am

leo_t_nice wrote:Also, this warning is caused by a preceeding stage having the "Preserve Partitioning" flag set to "Propagate". This is the default value, and tells PX that it should attempt to maintain a partitioning method. Obviously the sequential stage can't preserve the partitioning. If this option is set to "Clear", the sequential stage wont report a warning (i dont like to see warnings if i can stop them :D )

The problem is some stages don't have this option, such as DB2 Connector to reject link. Without this option to clear, how do you clear this warning?The warning is so obvious, why it produces this warning? It looks so dummy.

ray.wurlod · Post by **ray.wurlod** » Mon Apr 13, 2015 4:12 pm

Try clearing in the immediately upstream stage.