Download Data from http source

rsunny · Post by **rsunny** » Mon Jan 31, 2011 1:03 pm

Hi ,

I have a scenario where i need to download the data from http server(www.abc.com) into a flat file. Can anyone tell me what are the stages that i need to use in order to get the data into a file.As i tried using with xml stages and i couldn't able to succeed.

Any help is really appreciated.

Thanks in adance

rsunny · Post by **rsunny** » Tue Feb 01, 2011 7:03 am

Hi ,

Can any one tell me how to get the data or is it not possible to get the data from http to file. Is thee any specific format that datastage is going to accept. I herd that we can get it through Java stages but other than that is there any other option available.

Any help is really appreciated

Thanks in advance

chulett · Post by **chulett** » Tue Feb 01, 2011 8:33 am

As far as I know, the Java Pack is the only way to go.

rsunny · Post by **rsunny** » Tue Feb 01, 2011 9:31 am

Hi craig,

Cant we use any of the Real time stages to get the data from http to file. If we are getting the data from http server, does this server should be having any specific format to load to file either with java pack or with any other.

Any help is really appreciated.

Thanks in advance

chulett · Post by **chulett** » Tue Feb 01, 2011 10:31 am

If the website has a service you can leverage, then the "real time" stages could be used. Never made any attempt to use the Java pack or source from an HTTP server so can't really answer the other question.

eostic · Post by **eostic** » Tue Feb 01, 2011 10:32 am

I haven't tried it in awhile, but assuming that the data is a stand-alone xml document, the xmlInput Stage could take a url for retrieval of the document. I can't promise it will work, but it's worth a try.

If it's not xml, then it will have to be data that is delivered via a formal Web Service, with WSDL, and then you would use Web Services Pack.....

or.... use JavaPack and write your own http call. I did one years ago --- it wasn't that difficult, but you do need to have at least a little bit of java experience to understand how to build and compile your java code and do to the http bits inside of it.

...years ago people used the Perl Stage, but that no longer exists.....

Ernie

chulett · Post by **chulett** » Tue Feb 01, 2011 10:38 am

eostic wrote:...years ago people used the Perl Stage, but that no longer exists.....

Sniff... now that I've used. Too bad it be dead.

rsunny · Post by **rsunny** » Tue Feb 01, 2011 10:45 am

Hi Ernie,

i tried with xml stages but for xml input stage we have two options either xml document or URL/FIle path. But if we check the option URL/FIle path where can we give the url path as the input doent have anything to specify the url path and so i couldnt able to work out wih that.

And Do we need to purchase Webservices pack seperately ?Can you explain me how we can deliver the data via web sevice using WSDL.

Thanks craig and ernie for your replies

Any help is really appreciated

Thanks in adance

shukla_krishna · Post by **shukla_krishna** » Fri May 13, 2011 4:59 am

Have you got solution for this issue

eostic · Post by **eostic** » Fri May 13, 2011 6:09 am

The best solution for "some data" (not xml) is to use JavaPack. Code your own HTTP retrieval and then parse it as you wish and write it to an output link.

Ernie