Page 1 of 1

Working with XML Output and Web Services Transformer

Posted: Tue Feb 09, 2010 2:35 pm
by dganeshm
Hi All,

I am loading data into IBM Maximo via web services transformer using WSDL call. In the current design if there are 100 records , it makes 100 calls to Web service and there by belting the server 100 times. (100 is just an example , records go as high as 2 million). So in those cases its really taxing the server.

So what I tried is to consolidate all records in a single xml and make just one call to the Web Service and load it to server. When I tried about 100 records it worked smoothly , but when I ran it for 7000 records the job is in hung state , still trying to load into MAXIMO. Not a DataStage issue but a MAXIMO limitation. What I currently want to do is to limit the number of records in a XML , so is there a way to limit it while creating XML from the XML output , if there are 1000 records , then 2 XMLs should be generated with 500 records in each of them.

Any help would be greatly appreciated.

Posted: Wed Feb 10, 2010 2:55 pm
by eostic
Hi Ganesh....

Great work so far....often times sites don't have a web service that supports the sending of an "array" of rows --- fortunately your Maximo target appears to suppor that just fine.....smart idea also to break it up....even if Maximo didn't have a limit, you might eventually bump into DataStage's own upper limits............

Take a look at the "trigger" capability on the xml output stage. This allows you to select a column on the input (it doesn't have to be a column that will appear in your xml document) that "triggers" the release of XML on the output link. You will have to have several counters for the rows in an upstream stage, one that counts up to 500 and another that counts "units" of 500. Ultimately, that second counter will become your "trigger."

Ernie

Posted: Wed Feb 10, 2010 4:09 pm
by dganeshm
Hi Ernie,

That worked like a charm , although instead of using two counters , I did a mod (@INROWNUM,500) = 0 then count +1 else count , to keep track of every 500 rows.. Thanks again Ernie..

Posted: Wed Feb 10, 2010 11:05 pm
by eostic
Nice! ; )