COnver Server routine to parallel

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
shrey3a
Premium Member
Premium Member
Posts: 234
Joined: Sun Nov 21, 2004 10:41 pm

COnver Server routine to parallel

Post by shrey3a »

Gurus,

I've written a server routine which I wanted to convert to parallel routine.
We have c compiler installed on box , I wanted to wrap the below code in c shell and ompile it but I'm not sure abt the syntax i.e. can below code be just included in c shell as it is ..i tried to search for examples but got not luck ...below is the code....if anybody can help on this.

**************************************************
cmd = "cd /opt/isproj/ctl/ && ./ctlAccess" :" " :Arg1
Call DSExecute("UNIX",cmd,Output,SystemReturnCode)
Temp=EReplace(Output, @FM, "~")
Ans= Trim(Temp)
Ans = Field(Temp,'~', 3)
**************************************************

Regards,
Munish
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

First problem: you need a C++ compiler, not a C compiler. Do you have one?

Second problem: have you given any thought to what happens when your command runs multiple copies in parallel? Will it work properly?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
shrey3a
Premium Member
Premium Member
Posts: 234
Joined: Sun Nov 21, 2004 10:41 pm

Post by shrey3a »

Hi Ray,

We have c++ compiler and our job will be running in in sequential mode i.e. we have MQ Websphere plugin as source and messages are generated in order like before and after image comparison .....we have created the POC and it works ...we need to get the value from ctl tables on basis of argument passed...it works fine in server routine ..i tried to use the basic transformer but it failed. the only option I had was to create parallel routine for the below code and compile it thru c++ compiler I've never created the parallel routine before as external function.

Thanks
ray.wurlod wrote:First problem: you need a C++ compiler, not a C compiler. Do you have one?

Second problem: have you given any thought to what happens when your command runs multiple copies in parallel? Will it work properly?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Where / how often do you call this? Seems to me you may be able to leave it as and call it from a Sequence job, depending on the answer.
-craig

"You can never have too many knives" -- Logan Nine Fingers
shrey3a
Premium Member
Premium Member
Posts: 234
Joined: Sun Nov 21, 2004 10:41 pm

Post by shrey3a »

Hi,

I even proposed it to call from sequencer and pass the routine value to parallel job....but we can not use sequencer as our job source is MQ websphere and the way its configured it will pick up the messages(segments) from UDB log i.e. as soon as UDB table are updated the message will be published in MQ and ETL job whose source is MQ websphere will parse it to process the segment.....so there is no sequencer involved.

Regards,
chulett wrote:Where / how often do you call this? Seems to me you may be able to leave it as and call it from a Sequence job, depending on the answer.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Why not just leave it as a server job? The startup overhead is much less, which means that the output would arrive sooner.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
shrey3a
Premium Member
Premium Member
Posts: 234
Joined: Sun Nov 21, 2004 10:41 pm

Post by shrey3a »

ray.wurlod wrote:Why not just leave it as a server job? The startup overhead is much less, which means that the output would arrive sooner.
In server job MQ websphere stage is not available and also IBM has developed some segment splitter and product called DTS which we are using and its only available in parallel job pallette.

Regards,
pneumalin
Premium Member
Premium Member
Posts: 125
Joined: Sat May 07, 2005 6:32 am

Post by pneumalin »

To convert your basic routine to C routine is doable. The first two lines are ok via using strcat and system in C, but you may have to dig into the include files in .../DSEngine/include or .../PXEngine/include to find out if those functions such as trim, Ereplace/Convert, Field are declared, and search the binary .so to find out if these functions exist. I never got a time to search it, and please let me know if you get it to work.

Since you want to run the job in sequencial, why don't you keep exploring the Basic Transformer option if you find the C code is too much to do. You can post the failing message in your job here, I may be able to help on it.
Post Reply