DSXchange

Problem:
I need to download a file(using datastage) from the internet using HTTP protocol. Please let me know which stage can be used for this purpose.

eg)
URL: http://news.google.com/news

In the above example, I need to download the data found under the URL to a file on my machine.

Thanks,

JavaPack is an easy way to do it, assuming you are comfortable using HTTP within Java and then can spend some time building an interface for JavaPack ....I've also heard of people using PERL for this as well from within DataStage, although I haven't done it.......I could be wrong, but I seem to remember this being discussed relative to Java another time...maybe someone has built a class for this already.

Ernie

Perhaps something employing the 'wget' command?

For text files you could employ the external source stage with a 'wget {URL}; echo {file}' combination to fetch from the web then echo the file.

In other words, there's no built-in stage for this.

Thanks Everyone. I am planning to go with wget for now. I did not feel very comfortable as it is not available by default on solaris 10.