Page 1 of 1

webservice tranformer stage

Posted: Thu Oct 24, 2013 1:36 pm
by peep
we are using web service transformer stage to load data into mdm database.

Our datastage job which is using webservice transformer stage is processing very slow less than 10 row/sec.

our cpu ,memory , disk through out is absoutely at minimal usage.
let me know if there is any thing we can do to process messsages using WS transformer stage faster.

Posted: Thu Oct 24, 2013 1:50 pm
by ray.wurlod
You need to test the web service itself from some other kind of client to determine whether there's a limit on the rate of updates that the service itself can perform.

Posted: Fri Oct 25, 2013 9:22 am
by eostic
Web Services aren't a speedy protocol. Lots of things on both sides to deal with xml.

However...a couple of things you can look at....

a) as noted by Ray, see how the web service performs using another tool, to isolate any issues.

b) see if the target server can handle parallel streams of services being invoked....either from one Job (normal parallelism and use a multi-node config), from one Job and multiple paths formally drawn on the canvas, or perhaps simply multiple Jobs, either formally developed and named or using multi-Job instancing. All of these are options if the Server allows parallel input and doesn't care about the order of the services being called and the data rows that they are associated with...

c) How many columns? Is it a huge number? (many hundreds?).....and/or are you using xmlOutput or xmlStage to build a giant array for this service? There may be some things you can try if the volume of data per call is huge or if the number of columns is huge.

Ernie