I need help using web services in DataStage 8.7. I am trying unsuccessfully to connect to a remote Web Service using an https connection. I can connect to the service using SOAPUI but can't connect using DataStage. Thanks in advance for any help/advice that anyone may be able to provide. I am receiving the following messages from DataStage Director:
Web_Services_Transformer_15,0: Error: 0026`:`com.ascentialsoftware.wsclient.ServiceInvocationException: Service invocation exception: <ns1:Fault xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/">
<faultcode>soapenv:Server.userException</faultcode>
<faultstring>java.lang.RuntimeException: No support for attachments</faultstring>
<detail>
<ns1:stackTrace xmlns:ns1="http://xml.apache.org/axis/">java.lang.RuntimeException: No support for attachments
at org.apache.axis.Message.setup(Message.java:336)
at org.apache.axis.Message.<init>(Message.java:234)
at org.apache.axis.transport.http.HTTPSender.readFromSocket(HTTPSender.java:664)
at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:128)
at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:71)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:180)
at org.apache.axis.client.Call.invokeEngine(Call.java:2564)
at org.apache.axis.client.Call.invoke(Call.java:2553)
at org.apache.axis.client.Call.invoke(Call.java:1753)
at com.ascentialsoftware.wsclient.OperationCall.invoke(OperationCall.java:284)
at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1685)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1591)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1544)
at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:105)
at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)
at com.ascentialsoftware.jds.StageFactory.runTransformer(StageFactory.java:361)
</ns1:stackTrace>
</detail>
</ns1:Fault>
at com.ascentialsoftware.wsclient.OperationCall.handleRemoteException(OperationCall.java:387)
at com.ascentialsoftware.wsclient.OperationCall.invoke(OperationCall.java:292)
at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1685)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1591)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1544)
at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:105)
at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)
at com.ascentialsoftware.jds.StageFactory.runTransformer(StageFactory.java:361)
Web Services error
Moderators: chulett, rschirm, roy
If it works in SOAP UI, what is it trying to return? is it composing a file --- trying to send a lot of data? Is that data coming inside of a regular text message in the payload of the SOAP response? Or some other way?
SOAPui will show you the contents of the returned SOAP body....this may tell you what it looks like and what it is trying to accomplish.
Ernie
SOAPui will show you the contents of the returned SOAP body....this may tell you what it looks like and what it is trying to accomplish.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Re: Web Services error
Thanks for your fast response. With my very limited experience of web services, I will try and answer your questions as best as I can. It appears that it is trying to return a zipped xml file. Following is a portion of the SOAP body:
</xsd:complexType>
-<xsd:complexType name="someFileName1">
-<xsd:sequence>
<xsd:element name="return" type="xsd:base64Binary" ns2:expectedContentTypes="application/x-gzip" minOccurs="0" xmlns:ns2="http://www.w3.org/2005/05/xmlmime"/>
</xsd:sequence>
</xsd:complexType>
The dataset shouldn't be very large at all (maybe a few thousand records) with the exception of one file which may contain around 1 million records.
I am not sure on your question about if the data coming inside of a regular text message in the payload of the SOAP response.
If you need me to send other portions of the SOAP body I will be glad to do so.
Thanks,
Mark
</xsd:complexType>
-<xsd:complexType name="someFileName1">
-<xsd:sequence>
<xsd:element name="return" type="xsd:base64Binary" ns2:expectedContentTypes="application/x-gzip" minOccurs="0" xmlns:ns2="http://www.w3.org/2005/05/xmlmime"/>
</xsd:sequence>
</xsd:complexType>
The dataset shouldn't be very large at all (maybe a few thousand records) with the exception of one file which may contain around 1 million records.
I am not sure on your question about if the data coming inside of a regular text message in the payload of the SOAP response.
If you need me to send other portions of the SOAP body I will be glad to do so.
Thanks,
Mark
That's what I figured. There's no easy to do this with the WS Stages...they haven't been updated to formally support "SOAP with Attachments"....an addition to SOAP that gained popularity early on, but then drifted off (for various reasons)....haven't seen or heard the request in quite awhile.
You "might" be able to craft something yourself, but it will take a deeper understanding of SOAP payloads and then xml, especially xml containing some degree of "escaped" data that represents the "file" that is being sent along.
Goal one would be to see if you can properly receive the whole response. I do not know if you will be successful, but here's a strategy for approaching it:
Start with an example that passes a really small file. We can talk about how you might handle bigger ones later, but for initial functional testing, start with a little one.
In your WSStage's "output link", put ONE SINGLE column. Let's call it SOAPbody, and make it longvarchar with a length of (say) 999999. Put a single "/" in the Description property.
In the "Message" Tab for the output link, find the pull down that says something like "User Defined Message", and select your one single column.
Send that to a Sequential Stage...use "None" for Delimeter and for Quote Character.
See what you get....re-spinning your file from there may or may not be do-able within DataStage. I haven't seen a payload from a SOAP with Attachments service in a very long time. I "think" it used to be sent back in base64 or such, but it may have a different encoding scheme.
Then you "may" be able to en-encode that with DataStage, or you may be best using something external to do it. Hard to say at this point.
If it gets really large, you may need to start playing with the APT variables that deal with TRANSPORT BLOCK Size.....or consider using a Server Job, which is much more liberal in the passing of large strings in a single column on a link.
Good luck and keep us posted.
Ernie
You "might" be able to craft something yourself, but it will take a deeper understanding of SOAP payloads and then xml, especially xml containing some degree of "escaped" data that represents the "file" that is being sent along.
Goal one would be to see if you can properly receive the whole response. I do not know if you will be successful, but here's a strategy for approaching it:
Start with an example that passes a really small file. We can talk about how you might handle bigger ones later, but for initial functional testing, start with a little one.
In your WSStage's "output link", put ONE SINGLE column. Let's call it SOAPbody, and make it longvarchar with a length of (say) 999999. Put a single "/" in the Description property.
In the "Message" Tab for the output link, find the pull down that says something like "User Defined Message", and select your one single column.
Send that to a Sequential Stage...use "None" for Delimeter and for Quote Character.
See what you get....re-spinning your file from there may or may not be do-able within DataStage. I haven't seen a payload from a SOAP with Attachments service in a very long time. I "think" it used to be sent back in base64 or such, but it may have a different encoding scheme.
Then you "may" be able to en-encode that with DataStage, or you may be best using something external to do it. Hard to say at this point.
If it gets really large, you may need to start playing with the APT variables that deal with TRANSPORT BLOCK Size.....or consider using a Server Job, which is much more liberal in the passing of large strings in a single column on a link.
Good luck and keep us posted.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Re: Web Services error
Sounds like we will be getting our feet wet with SOAP and XML very quickly...
Thanks for the information and we will try your suggestions over the next few days. We will be sure to let you know what we come up with...
Mark
Thanks for the information and we will try your suggestions over the next few days. We will be sure to let you know what we come up with...
Mark
Re: Web Services error
Ernie,
Based on what we have learned so far, we have decided to try a different route to get the file attachments from the web service. We are considering on going with either JAVA or PERL and would be interested to hear your thoughts on the advantages/disadvantages of each.
Thanks,
Mark
Based on what we have learned so far, we have decided to try a different route to get the file attachments from the web service. We are considering on going with either JAVA or PERL and would be interested to hear your thoughts on the advantages/disadvantages of each.
Thanks,
Mark
I can't comment on Perl --- haven't done enough with it....but with Java, the Java Integration Stage (or Java Transformer if you are on an older release than 9.1) let's you "talk" to the DS Engine at a very deep level. You "feed" rows to it after you parse something, or do whatever you want to do inside of that Java program.
...if the goal is to just call the service, drop the file, and then have DS read and parse that file, then go with whatever language is easiest (your or your team's available skill sets) and invoke it as a program prior to the DS Job.
...if the goal is to call the service "for every row" that comes from an upstream source, and then run continually, parsing the returned "file" on the fly and doing things with it before getting the "next row" and making the "next invocation of the service" (and the job never stopping between all those invocations), then go with the Java scenario and integrate it tightly into your DS Job and solution.
Ernie
...if the goal is to just call the service, drop the file, and then have DS read and parse that file, then go with whatever language is easiest (your or your team's available skill sets) and invoke it as a program prior to the DS Job.
...if the goal is to call the service "for every row" that comes from an upstream source, and then run continually, parsing the returned "file" on the fly and doing things with it before getting the "next row" and making the "next invocation of the service" (and the job never stopping between all those invocations), then go with the Java scenario and integrate it tightly into your DS Job and solution.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Re: Web Services error
Our requirements are more like the first scenario where we will call the service, drop the file and then have DS read and parse the file. I will recommend that our programmers use whatever language they are more comfortable with. Thanks again for all of your help.