Hierarchical Data - Issue with mapping in REST Step

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
jhmckeever
Premium Member
Premium Member
Posts: 301
Joined: Thu Jul 14, 2005 10:27 am
Location: Melbourne, Australia
Contact:

Hierarchical Data - Issue with mapping in REST Step

Post by jhmckeever »

Hi,

I am trying to fetch data coming from a RSS Feed -

http://www.theage.com.au/rssheadlines/b ... le/rss.xml

Below are the steps I have followed -

1. Choose hierarchical data stage to perform the operation.
2. Go to properties of hierarchical data stage, and click on Edit Assembly.
3. Add Rest step from the pallett in Assembly Editor.
4.1. In configuration - General, choose GET method as HTTP method.
4.2.. Provide url as http://www.theage.com.au/rssheadlines/b ... le/rss.xml
4.3. HTTP version as 1.1
5. Security - None
6.1. In Configuration - Request - Check load the outgoing body from.
6.2. Choose text node body with content type as application/xml.
6.3. Choose Encoding type as UTF-8
7. Response is same configuration as request
8. In mappings, I add #rss# into the source and then map it with body.
9. Configuration is now complete.

When I go into test data tab, CDIUI2501I Either no Output data is available, or you need to test the assembly before Output data can be displayed. I did curl on the mentioned url and I am receiving data. I think the issue is somewhere with mapping.

Could someone please guide what am I doing wrong? I tried fgoogling this issue as well as finding out the root cause of the problem So far no luck!

Regards,

John
<b>John McKeever</b>
Data Migrators
<b><a href="https://www.mettleci.com">MettleCI</a> - DevOps for DataStage</b>
<a href="http://www.datamigrators.com/"><img src="https://www.datamigrators.com/assets/im ... l.png"></a>
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

I've been working with RESTful GETs and the hierarchical stage for a while now, and after some pain getting starting it is working now.
We usually have a REST step followed by JSON parsers and then the output step.

What do you have as the Configuration -> response setting for content type? (I assume you are choosing the first option to output a text node named "body")

Did you hit the button "Test Assembly" at the top of window before going to "Test Data"?
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

....another key thing to do is to first get a perfectly working Stage that uses a disk version of your payload.

With REST, you have two challenges --- exactly how to specify the call and invokation...and get the URL and the security correct....

...and how to parse the returned payload.

Separate the two. Save the payload with curl to a normal xml document on disk. One that you can open successfully in your browser as a regular xml doc. Write the JOb that parses that perfectly, so that you are fully familiar with the mappings you need.

Then work on the REST steps. Just map the output to a single output column to "get it out" as a full xml chunk. There are many ways to do this...one is to right click in the "root" selection dialog and choose the chunk option.....see if you are getting any payload at all......

....then adapt this assembly to the one you proved out when you read the xml or json from disk.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply