DS Webservice Transformer Not processing all requests

ami · Post by **ami** » Wed Mar 05, 2014 5:47 pm

Hi,

I'm trying to make a call to a web service for multiple inputs and process the response(s) received. My job design is -

Input Sequential File (containing all inputs as per _IN of WSDL and there is only one column) -> XML Output (All rows aggregated - single output column) -> Copy Stage -> Web Service Transformer (Response output is XML; user defined message for both input and output) -> Copy Stage -> XML Input (parsing as the _OUT of WSDL) -> Sequential File output.

Prior to adding the copy stage the job kept on failing on the transfomer saying (content not allowed in prolog). After adding the copy stage(s) the job seems to be running fine but I'm getting response for first input row only. I have checked the other inputs separately and they are running OK.

This is my first attempt at calling web service so quite possibly I'm missing something here - but have no idea what! Can anyone please help me out?

eostic · Post by **eostic** » Wed Mar 05, 2014 6:34 pm

Sounds pretty strange. Not sure what it might be, but this is one of those times that I might suspect something with timing between operators and for testing and/or for the ultimate solution, would suggest building it as a Server Job. Same Job structure --- just use regular Transformers in the place of your Copy Stages (really, it's just for a separator so that your xpaths don't get tangled up between your xmlOutput and WSTransformer and xmlInput Stages). Everything else should be the same.

Even before that, make sure you are running in a single node config on the parallel side.

Ernie

ami · Post by **ami** » Wed Mar 05, 2014 8:02 pm

Thanks Ernie.

I thought I had a revelation when you said make sure to run in one node. But no no change in output. I then realized the copy stages were in parallel so changed that to sequential. No change in output.

Finally implemented the job as Server job and no change in output. I took an output of the XML output stage and it looks like

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<!--
  - Generated by Ascential Software Corporation, DataStage - XMLOutput stage - 
  - Thu Mar 06 12:12:32 2014
  -->
<ns1:LookupValue xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ns1="http://tempuri.org/" xmlns:ns2="http://schemas.datacontract.org/2004/07/Lookup.Services.Contracts.Entities">
  <ns1:inputParam>
	ABCD1
  </ns1:inputParam>
  <ns1:inputParam>
	ABCD2
  </ns1:inputParam>
  <ns1:inputParam>
	ABCD3
  </ns1:inputParam>
  <ns1:inputParam>
	ABCD4
  </ns1:inputParam>
  <ns1:inputParam>
	ABCD5
  </ns1:inputParam>
</ns1:LookupValue>

Is this is a valid input for the transformer to process multiple inputs?

eostic · Post by **eostic** » Wed Mar 05, 2014 8:57 pm

It might not be.

Get yourself a copy of SOAPui or other web services testing tool. If the Web Service TRULY supports arrays, then SOAPui will respond and allow you to enter multiple values.......

...capture that XML (SOAPui will show it to you).....

...THAT xml is what your xmlOutput Stage needs to create in order to pass the multiple values in.

Good debugging.....you may have hit exactly what the issue is.

Ernie

ami · Post by **ami** » Thu Mar 06, 2014 1:21 am

Thanks Ernie. Yeah seems that's the case. SOAPUI is taking one at a time, doesn't like all at once. But the SOAP developers are gone for the day - can't check it until morning.

So if I end up putting one request at a time then can I just avoid aggregating - what do I need to change in the current job design.

I tried putting directly of the source in to the transformer and it aborted. But I could have easily missed some config.

eostic · Post by **eostic** » Thu Mar 06, 2014 3:39 am

ah....well...if SOAPui didn't automatically recognize and allow you to enter more than one value, then perhaps the web service is not specifically set up to permit an array ---- you can't send an array unless the WSDL actually permits it.........

Ernie

bikramg · Post by **bikramg** » Sat Mar 08, 2014 4:52 pm

Hi Ernie, thanks for your response. This is my other \ individual account - my group account all of a sudden giving me login issue, saying inactive or something. Tried contacting the webmaster twice but no response so not sure what's going on. Anyway, I couldn't read your entire message (as this is ordinary account) - but tried couple of things - seems to be working.

In the XML output stage I change the aggregate to "Use Trigger Column" and it seems to process all requests individually. However I noticed the output isn't deterministic. In first run it processed 16 requests (of 20 in the source file) in the second run it processed 16 as well but there were couple of different records. Any ideas why this could be happening - seems web service timing to me but again there could be something I'm missing.

Thanks again. Hopefully my group account will be sorted soon so that I can read your reply.