Working of Link Ordering

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Working of Link Ordering

Post by dxk9 »

Hi,

I have a job which reads data from DB2.. Upserts them into DB2 (link 1) & update a flag in DB2(link 2).

How this works ?? Will the 1st record sent to Link 1 gets processed before its being sent to link 2 ??

Or the transformer link ordering doesn't have any impact on this & the records flow directly on link 1 & link 2. (both are main outputs & not rejects).

Regards,
dxk9
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Working of Link Ordering

Post by chulett »

dxk9 wrote:Will the 1st record sent to Link 1 gets processed before its being sent to link 2 ??
This.
-craig

"You can never have too many knives" -- Logan Nine Fingers
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

We don't have any delay in the processing of records in Link1 (insert else update) & Link2 (update). Especially don't want Link 2 to be faster than Link1.

But since the logic of both the links are different, we thought it might be an issue.

So since you say that Link1 processes first before the record goes to Link2, we can go ahead with the design right ?

Thanks,
dxk9
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

To add on.. The array size & Transaction size of both are (1,200).

Thanks,
dxk9
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It really is as simple as the links are completely processed in order, so the first one must go to the target and its fate determined before the second one is... released.

Now, array and transaction sizes can complicate error reporting and may affect how it works depending on the relationship between the two links. If the second link depends on seeing the result of the first link then you may need to drop one or both of the sizes down to 1 or at least ensure both links are sent to the same (single) target stage so their work is done in the same 'transaction'.
-craig

"You can never have too many knives" -- Logan Nine Fingers
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

Thanks Craig !!

But I got only your 1st statement, wasn't able to read the rest of them as its a 'premium content' :(

Thanks,
dxk9
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

As Ray would say, the issue of Premium Content is easily solved. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

Its not free.. so its not easy :(
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Without it, DSXchange would cease to exist.

Premium memberships are used to fund the hosting and bandwidth charges incurred by DSXchange. Notice the absence of ads? Notice the good signal to noise ratio?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

Very true Ray.. !! I agree to ur point..
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

also check interprocess row buffering...that will dramatically impact the normal default "process each link thru completion" behavior.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Thanks Ernie... I had made a mental note to mention that as well and then managed to forget it mid-post, it would seem. :(
-craig

"You can never have too many knives" -- Logan Nine Fingers
rameshrr3
Premium Member
Premium Member
Posts: 609
Joined: Mon May 10, 2004 3:32 am
Location: BRENTWOOD, TN

Post by rameshrr3 »

I guess a company can apply for a group membership ? Or is that feature not available yet ? After all no biz says no to 'help' that leads to profit$
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Corporate discounts are available, but the model is for multiple individual memberships from that company's email domain. From memory the discounts kick in at five users.

But even without the discount it's less than 30c (Rs12) per day. A week on DSXchange costs less than a cup of coffee!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
dxk9
Participant
Posts: 105
Joined: Wed Aug 19, 2009 12:46 am
Location: Chennai, Tamil Nadu

Post by dxk9 »

How to check on the 'interprocess row buffering' and what kind of impact does it hold on the behaviour..

I read through this, but since I couldn't test it to really understand the flow, felt your thoughts might be more helpful in understanding the concept.

Thanks,
dxk9
Post Reply