Force Sequential mode

[m_mani87]

Hi my query is about force sequence mode in Job Parameters

The Job structure
Seq file----------> Transformer-------------------->Seq file
I ran the above job in 3 ways

1) Transformer in sequence -> Got Output A
2) Transformer in Parallel -> Got Output B
3) Force sequence mode enabled and Transformer Parallel -> Got Output B

I am clear with design 1) and 2)
But in design 3) since the force sequence mode is enabled why it didnt make the job to run sequentially. Ideally i should have got output A(though the tansformer in parallel) due to Force Sequential mode. Why it is returning output B instead of output A

Kindly help on this as this would clarify my various doubts

[priyadarshikunal]

Would be easier to explain if you describe the 3rd design, along with that, what tasks you are performing it the transformer?

[ArndW]

How are "Output A" and "Output B" different? Different order? Different number of rows? Different content?

Are you doing any sort of multi-row processing using stage variables in your transform stage or are you passing the data directly from input to output (with some transformations)?

The only thing that could be different is the ordering, if you sort A and sort B and then compare the files they will be identical regardless of how many partitions you use (assuming you are doing a 1:1 transfer in the transform stage.

[m_mani87]

Input file:
empno
10
20
30
40
50

The Stage variables i used in Transformer

StageVar+ empno --- > StagegVar1
empno ----> StageVar (Intial value is 10)

Im taking output StageVar1

1) Transformer Sequentially output:

empno StageVar1
10,20
20,30
30,50
40,70
50,90


2) Transformer Parallel output:

10,20
50,70
40,50
30,40
20,30

The output will be obviously diff as the initial value set as 10 for second variable is node specific.
Kindly clear on the 3rd design

[ArndW]

Do you mean "Compile in Trace Mode" and then "Force Sequential Mode"?

I think that this isn't what you want, you need to go to the transform stage in the Designer, do Stage Properties -> Advanced -> and set the execution mode to "sequential".

[m_mani87]

ArndW,

You are right. I wanted to run the transformer sequentially to achive desired output.

I thought of searching for some thing in job level to make the job run sequentially.
But now i find that Force sequential mode in Job properties and Execution mode set to sequence in transformer are different.

Could you explain what is the difference between both. Is there any way to make the same work in Job level as well.

[ArndW]

I haven't used that debug/trace method, so haven't looked into detail at what it does "under the covers". Generally one should have a parameter to all parallel jobs for the APT_CONFIG_FILE and by using a 1-node configuration you can ensure that stages run sequentially. I don't know if there is a DSPARAMS global setting in addition to the APT_CONFIG_FILE which will force sequential mode, I kind of doubt it exists and I certainly haven't come across it.

Did you use the stage-level force sequential mode and did it work? If yes & yes you could close this thread off a resolved.

[m_mani87]

Yes ArndW I have used Stage level sequential mode in transformer and it worked. But my question from the beginning as the subject was what this Force sequence mode does in job parameters.

[ArndW]

The Job Parameters don't contain that, the job properties do.

According to help, the sequential setting applies only to the conductor node. I couldn't find more information on the trace mode, but it probably applies to the c++ compiler and loads in more stack- and variable- tracing during compile and linking (allowing use of dbx and perhaps more details truss output).