DATASTAGE WEB SERVICES PACK V7.5

[vsi]

Hi All

I am Back with my problem again.

I have a job design like
webservices client->xml Input->seq file->transformer->oracle

This job is running fine form past few month, today we are tried to test large volume of data, then all of sudden my jobs got aborted and gives me the bellow warning and error messages.

could you please let me know why am i getting this error

I have attached the error message for your reference


Message:warning
jobJI..XML_Input_JUV: java.lang.OutOfMemoryError


Message:fatal
jobJ..XML_Input: java.lang.OutOfMemoryError
at org.apache.axis.message.SAX2EventRecorder$objArrayVector.add(SAX2EventRecorder.java(Compiled Code))
at org.apache.axis.message.SAX2EventRecorder.endElement(SAX2EventRecorder.java(Inlined Compiled Code))
at org.apache.axis.encoding.DeserializationContextImpl.endElement(DeserializationContextImpl.java(Compiled Code))
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at javax.xml.parsers.SAXParser.parse(Unknown Source)
at org.apache.axis.encoding.DeserializationContextImpl.parse(DeserializationContextImpl.java:242)
at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:538)
at org.apache.axis.Message.getSOAPEnvelope(Message.java:376)
at org.apache.axis.client.Call.invokeEngine(Call.java:2583)
at org.apache.axis.client.Call.invoke(Call.java:2553)
at org.apache.axis.client.Call.invoke(Call.java:1753)
at com.ascentialsoftware.wsclient.OperationCall.invoke(OperationCall.java:266)
at com.ascentialsoftware.wsclient.WebService.callServiceInternal(WebService.java:1659)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1566)
at com.ascentialsoftware.wsclient.WebService.callService(WebService.java:1519)
at com.ascentialsoftware.wspack.Transformer.process(Transformer.java:102)
at com.ascentialsoftware.jds.StageFactory.runStage(StageFactory.java:544)
at com.ascentialsoftware.jds.StageFactory.runSource(StageFactory.java:335)


Thanks

[eostic]

Hi...

Chances are, the size of the returned SOAP envelope has grown and you are now being sent a much larger quantity of data.

We won't debate the subject of "large" SOAP envelopes here, but unless there's something else going on, you should be able to make progress on this issue by increasing the heap size for the JVM used by Web Services Transformer.

After you open your Job, right mouse click on the WSTransformer Stage and select "grid style". This will show you all the properties of the Stage that you normally see in the GUI, along with one other that is otherwise (not sure why) hidden.

Find the JVM Options property and put in -xmx500m (or some other numeric - this is just an example) until things work. You can look up the JVM options in google or other places. This is for mem-max specification.

The property is at the Stage level and the link level -- you may have to play with it to determine what's most appropriate to use.

Let us know if this helps at all with the symptom you are having.

Ernie

[dilip.datastage]

HI,
Can you help me like how and from where i can download web services pack, my default installation does not contain this service

Thanx in Advance,
Dilip

[ray.wurlod]

Send money for free receipt.

The Web Services PACK is separately licensed.

[vsi]

Thanks for your timely response


we are not using webservices transformer in our Job design.

we are using webservices client (source) stage to connect URL from there we are picking up data.

Thanks in advance

[chulett]

So? Because of that you are completely discounting Ernie's response? Simply substitute 'WSClient' everywhere he said 'WSTransformer', then go try out his suggestion.

[eostic]

Thank you Craig.

Per the WSClient, I can't remember if it also has the JVMOptions in the grid style editor, but I suspect it does....and it if doesn't, just use the WSTransformer instead of the WSClient. Only difference is that you'll have to fake up a way to send it a single dummy input row to kick things off.

...for those not familiar with the two stage types, you typically use the WSClient for Services that are "output only" or "input only" and called once for the job, as opposed to the WSTransformer which is something you want to invoke for "every row" that goes thru it.

Ernie

[eostic]

Thank you Craig.

Per the WSClient, I can't remember if it also has the JVMOptions in the grid style editor, but I suspect it does....and it if doesn't, just use the WSTransformer instead of the WSClient. Only difference is that you'll have to fake up a way to send it a single dummy input row to kick things off.

...for those not familiar with the two stage types, you typically use the WSClient for Services that are "output only" or "input only" and called once for the job, as opposed to the WSTransformer which is something you want to invoke for "every row" that goes thru it.

Ernie

[vsi]

Hi...

Chances are, the size of the returned SOAP envelope has grown and you are now being sent a much larger quantity of data.

We won't debate the subject of "large" SOAP envelopes here, but unless there's something else going on, you should be able to make progress on this issue by increasing the heap size for the JVM used by Web Services Transformer.

After you open your Job, right mouse click on the WSTransformer Stage and select "grid style". This will show you all the properties of the Stage that you normally see in the GUI, along with one other that is otherwise (not sure why) hidden.

Find the JVM Options property and put in -xmx500m (or some other numeric - this is just an example) until things work. You can look up the JVM options in google or other places. This is for mem-max specification.

The property is at the Stage level and the link level -- you may have to play with it to determine what's most appropriate to use.

Let us know if this helps at all with the symptom you are having.

Ernie

Erine,

Thanks for the reply. I have tried by giving -Xmx1000 in my job properties. But it thows me a different error

I have posted the error bellow for your reference


Project:DM
Job name:jobJI
Event #:46
Timestamp:2/15/2008 2:16:55 PM
Event type:Fatal
User:dscon08
Message:
jobJIM..Web_Services_Client_Juv: TJClient::initialize: unable to create Java Virtual Machine; classpath = /etl/Ascential/DataStage/DSEngine/java/lib/wsclient.jar
/etl/Ascential/DataStage/DSEngine/java/lib/axis.jar
/etl/Ascential/DataStage/DSEngine/java/lib/commons-discovery.jar
/etl/Ascential/DataStage/DSEngine/java/lib/commons-logging.jar
/etl/Ascential/DataStage/DSEngine/java/lib/jaxrpc.jar
/etl/Ascential/DataStage/DSEngine/java/lib/saaj.jar
/etl/Ascential/DataStage/DSEngine/java/lib/wsdl4j.jar

Could you please see this let me know if you have any idea

[chulett]

I have tried by giving -Xmx1000 in my job properties.

These things are usually case-sensitive and Ernie noted you should use -xmx not -Xmx. Unless that's just a typo in your post, try again with all lower-case and see if the error changes.

ps. Ernie - that property is there in the WSClient as well, I checked before I posted.

[lstsaur]

Actually, -Xmx is correct syntax to increase heap size.

[chulett]

Good to know, then... never mind.

[vsi]

I have tried by giving -Xmx1000 in my job properties.

These things are usually case-sensitive and Ernie noted you should use -xmx not -Xmx. Unless that's just a typo in your post, try again with all lower-case and see if the error changes.

ps. Ernie - that property is there in the WSClient as well, I checked before I posted.

I have tried by giving -Xms128m then it created two file in default project directory and again throughs me an error like Out of Memory.
these are the file created on defult directory

JVMDG315: JVM Requesting Heap dump file

JVMDG318: Heap dump file written
to /etl/Projects/DM/heapdump507912.1203106865.txt

JVMDG303: JVM Requesting Java core file
JVMDG304: Java core file written to /etl/Projects/DM/javacore507912.1203107030.txt

JVMDG215: Dump Handler has Processed Outofmemory Signal -1.

Then i have changed property the value like -Xms256m then it throughs me an error like

Project:DM
Job name:jobJ
Event #:37
Timestamp:2/15/2008 3:47:16 PM
Event type:Warning
User:dscon08
Message:
Abnormal termination of stage jobJIMSGetJuvTroubleshooting3..XML_Input_Jdetected

Please let me know if you have any idea on this

Appriciated your respnse

Thanks

[eostic]

Tough call. I've only had to use it twice, with some really big soap envelope returns, and I can't recall if it was 7.0 or 7.5. It would be interesting to note whether:

a) your job just dies immediately, no matter what, when you put this value into the stage.
b) it works for "some" services and not others (...try it on another web service; and on an xmethods service).
c) when it dies, does it sit and wait for awhile? (while trying to parse the too-large envelope)
d) tracing still works --- do you get an output call in the log when you trace the WS stage and check all 4 boxes on the right --- in a working scenario you should get the entire soap body in the log messages for request and for response.

...it will help to be sure that it's still not a syntax issue, new anomaly, or specific value in -Xmx statement...

Ernie