Working of Link Ordering

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

It's a setting at the Job level (job properties) ...OR...might also be at the Project level (something I do NOT recommend). It implements pipeline-style parallelism by default for Server Jobs....there are quite a few threads in here on the subject if you do some searching.

As it relates to this particular thread, turning on pipeline style parallelism changes the flow because it sort of gives more autonomy to each of the Stages....they can all do some degree of work concurrently....and that means that a Stage with (say) 4 output links will share data down each of those links as the logic permits, instead of sending a row down the "first" output link and doing nothing more until that "path" has completed.

The default behavior of link ordering illustrates a place where Server can do some creative things that aren't as easy to do in EE....for example, being able to force an Insert (and absolutely know that it was performed) before doing a lookup in another "later" link path while still being driven by the same row that came in from the ultimate source. When volumes aren't rediculously high, and transactional control is a requirement, Server is a very specialized tool that intuitively provides this ability without jumping thru hoops or coding an alternative.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

PS.... I haven't tried it in quite a few releases, but I did a research project years ago where I traced the path of a single row and value, using the debugger, with breakpoints all over the place and then turning inter-process row buffering on or off.....it was very insightful, showing how and where rows were sent thru the Job, and what activities were performed when.

This discussion about link paths and their processing is especially critical in real-time Jobs where the timing of the activity performed on a row is as important, or more important, than the throughput --- especially when performance is not a concern because of low volume data or transaction quantities.

-ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply