Reading flat file data on third party web site

[rkumar28]

Hi,

I have a situation....where I have to make a [url]HTTPS://[/url] connection to the third party web site to grab 5 files from their server. and load the data into relational table using DataStage.

Can anyone please help me with the shell script for making this connection and grabing or reading the files from their server to our unix box.

I will really appreciate any help in this regards.

Thanks
Raj

[ray.wurlod]

Any reason you can't use an FTP stage?

One of the capabilities in UniVerse that is not in DataStage is the set of functions collectively known as CallHTTP. That is, DataStage does not have the capability natively.

I hope someone can help you with an operating system script. If it's a well-formed script, you might be able to use it as the filter in a Sequential File stage, and read from its output directly into a DataStage job.

[mandyli]

Hi,

The simple and easy way using FTP or FTP stage. through FTP you can easily loaded in to ANY database.

Try it ..

[rkumar28]

Hi,

We get this file from third party vendor secure website and they are not supporting FTP (ONLY HTTPS connection to their website).
Currently, we are converting all process from INDIGO(another java based ETL tool) to DataStage. Indigo currently has a capability of making HTTPS connection. Looks like with current version of datastage we cannot do this....

I am trying to figure out if we can do this in unix shell script.

[roy]

Hi,
well if you can get an API or even a short program be it scripts or any programing language that is localy on the DS server you can invoke it via command line thus getting the files to your DS server.
if you have a java code that does the job your in better shape then you thought, just build a short program, wrap it in a DSExceute and your done:).
in any case get your support on this, who knows they might realize that this kind of thing might be a growing demand and get the functionality in DS

IHTH,