Restful web service

abhinavsuri · Post by **abhinavsuri** » Fri Nov 13, 2015 3:41 pm

I am using a hierachical stage to call a restful webservice and am trying to write to sequential file. I configured the restful and general step but am getting below error. Anyone who could help me interpret this would be of great help

Hierarchical_POST,0: Fatal Error: 2015-11-13 16:32:34,571 Fatal [com.ibm.e2.applications.runner.E2AssemblyRunner] [] caught fatal exception
com.ibm.e2.core.exceptions.OperatorException: OperatorTransitionFailedAtRuntime operator = 'UserRuntimeOperatorFrame:Op-RT-1[REST]', cause = 'com.ibm.e2.util.logging.E2LoggerCreationException: error instantiating class com.ibm.e2.util.logging.jul.E2JULLogger'
at com.ibm.e2.core.exceptions.OperatorException$FactoryImpl.operatorTransitionFailedAtRuntime(OperatorException$FactoryImpl.java:115)
at com.ibm.e2.core.framework.runtime.OperatorController.runOperatorStep(OperatorController.java:72)
at com.ibm.e2.core.framework.runtime.scheduler.OperatorTask.runFrame(OperatorTask.java:86)
at com.ibm.e2.core.framework.runtime.scheduler.OperatorTask.execute(OperatorTask.java:40)
at com.ibm.e2.core.framework.runtime.scheduler.AbstractTask.run(AbstractTask.java:29)
at com.ibm.e2.core.framework.runtime.scheduler.TasksExecutorSingleThreaded.run(TasksExecutorSingleThreaded.java:38)
at com.ibm.e2.core.framework.runtime.scheduler.ProcessSchedulerBase.prepare(ProcessSchedulerBase.java:74)
at com.ibm.e2.applications.wisd.process.E2ProcessInstanceForWISDImpl.initialize(E2ProcessInstanceForWISDImpl.java:208)
at com.ibm.e2.applications.runner.E2AssemblyRunner.initializeAssembly(E2AssemblyRunner.java:314)
at com.ibm.e2.applications.runner.E2AssemblyRunner.run(E2AssemblyRunner.java:192)
at com.ibm.e2.connector.datastage.cc.CC_E2Adapter.run(CC_E2Adapter.java:713)
Caused by: com.ibm.e2.util.logging.E2LoggerCreationException: error instantiating class com.ibm.e2.util.logging.jul.E2JULLogger
at com.ibm.e2.util.logging.E2LoggerFactory.getLogger(E2LoggerFactory.java:106)
at com.ibm.e2.util.logging.E2Logger.getLogger(E2Logger.java:40)
at com.ibm.e2.provider.ws.common.RESTWSLoggerDelegate.setApacheHTTPLogger(RESTWSLoggerDelegate.java:73)
at com.ibm.e2.provider.ws.common.RESTWSLoggerDelegate.setApacheHttpLogging(RESTWSLoggerDelegate.java:48)
at com.ibm.e2.provider.ws.connector.RESTWSConnector.initialize(RESTWSConnector.java:96)
at com.ibm.e2.providers.xml.AbstractXMLProviderRuntime.initializeLocalState(AbstractXMLProviderRuntime.java:204)
at com.ibm.e2.core.framework.frames.UserRuntimeOperatorFrame.initializeLocalState(UserRuntimeOperatorFrame.java:134)
at com.ibm.e2.core.framework.frames.AbstractRuntimeFrame.initializeLocalRuntimeState(AbstractRuntimeFrame.java:1700)
at com.ibm.e2.core.framework.runtime.OperatorController.runAutomaticStateTransitions(OperatorController.java:414)
at com.ibm.e2.core.framework.runtime.OperatorController.runTransitions(OperatorController.java:97)
at com.ibm.e2.core.framework.runtime.OperatorController.runOperatorStep(OperatorController.java:66)
... 9 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at com.ibm.e2.util.logging.E2LoggerFactory.getLogger(E2LoggerFactory.java:93)
... 19 more
Caused by: com.ibm.e2.util.logging.E2LoggerCreationException: java.io.FileNotFoundException: /var/tmp/XMLStage_org.apache.http.wire_0.log (Permission denied)
at com.ibm.e2.core.debug.CircularFileLoggerHandler.<init>(CircularFileLoggerHandler.java:112)
at com.ibm.e2.provider.ws.runtime.RESTWSApacheHttpLoggerHandler.<init>(RESTWSApacheHttpLoggerHandler.java:37)
... 24 more
[:0]

chulett · Post by **chulett** » Fri Nov 13, 2015 3:52 pm

Looks like this is the relevant bit:

Caused by: com.ibm.e2.util.logging.E2LoggerCreationException: java.io.FileNotFoundException: /var/tmp/XMLStage_org.apache.http.wire_0.log (Permission denied)

qt_ky · Post by **qt_ky** » Sat Nov 14, 2015 7:16 am

Aren't the lengthy java errors just so wonderful??? They give everyone something to spend time wading through.

abhinavsuri · Post by **abhinavsuri** » Tue Nov 17, 2015 1:57 pm

OK I managed to fix that one by marking all the output columns as NULLable. NOw I am able to Run , Test Assembly and it completes succeffully. However, wehen I check the response it says the test failed with below error. How can I fix this? Any help would be appreciated.

success, errorMessage, faultHttpStatusCode, faultHttpBody, httpVersion, statusCode, reasonPhrase, body
"false","CDIER0410E: Error in step=REST, cause=java.lang.NoSuchMethodError: org.apache.http.client.methods.HttpRequestBase.releaseConnection()V","","",null,null,null,null

eostic · Post by **eostic** » Tue Nov 17, 2015 2:25 pm

Here some things look at:

1. are you able to invoke the exact REST url that you are interested in using your browser (browser uses "GET" by default)?

2. get that working --- assuming it works, save the xml payload (or json) that you receive. Put it into a file.

3. do you have an xsd for that payload? If not, generate one, possibly using tools like "trang" (open source...easy to find on the web).

4. parse that xml/json payload, writing the assembly that successfully pulls out whatever you need from that xml.

5. once you have been able to write an assembly that successfully parses the structure, create another Job with the stage that just tries to call the service. See details here:

https://dsrealtime.wordpress.com/2015/0 ... datastage/

....basically take this one step at a time....and build SEPARATE jobs to call your service (and just dump the payload), and another Job that parses the payload.... then put them together when you have the independent aspects working perfectly. It's too difficult to plan, design and debug both things at the same time.

Ernie

abhinavsuri · Post by **abhinavsuri** » Tue Nov 17, 2015 2:52 pm

Yes I am able to get it working in the browser. I cant see the second portion of your post

abhinavsuri · Post by **abhinavsuri** » Tue Nov 17, 2015 2:57 pm

This is the output of test assembly

<?xml version="1.0" encoding="UTF-8"?><top>
<InputLinks/>
<result isOutputEnrichmentRoot="true">
<GETResponse>
<callStatus>
<success>false</success>
<errorMessage>CDIER0410E: Error in step=REST, cause=java.lang.NoSuchMethodError: org.apache.http.client.methods.HttpRequestBase.releaseConnection()V</errorMessage>
<faultHttpStatusCode/>
<faultHttpBody/>
</callStatus>
<statusLine/>
</GETResponse>
</result>
</top>

abhinavsuri · Post by **abhinavsuri** » Tue Nov 17, 2015 2:59 pm

However, in the log I see below message

Peek_15,0: success:false errorMessage:CDIER0961E: The REST step is unable to invoke the REST service, cause=org.apache.http.conn.ConnectTimeoutException: Connect to welcome.coremetrics.com:443 timed out faultHttpStatusCode: faultHttpBody: httpVersion:NULL statusCode:NULL reasonPhrase:NULL body:NULL

ray.wurlod · Post by **ray.wurlod** » Tue Nov 17, 2015 3:55 pm

If the web server has timed out, you need to get in touch with them and get them to "feed the hamsters" that power their site.

cyclogenisis · Post by **cyclogenisis** » Wed Jan 06, 2016 12:34 pm

abhinavsuri wrote:However, in the log I see below message

Peek_15,0: success:false errorMessage:CDIER0961E: The REST step is unable to invoke the REST service, cause=org.apache.http.conn.ConnectTimeoutException: Connect to welcome.coremetrics.com:443 timed out faultHttpStatusCode: faultHttpBody: httpVersion:NULL statusCode:NULL reasonPhrase:NULL body:NULL

I am working with coremetrics API's myself. This appears to be coremetrics issue, are you sure your API url is correct and the report is available for the period your url is set for? Also the message previous to this one (ie. close connection) I was receiving in the hierarchal stage but when the job is run fully it is able to get data back fine.