Does ETL able to handle TCP Socket connectivity

[thirupri]

Hi Guru's,

Does ETL able to handle TCP Socket connectivity? Wherein it can send request message and receive reply (or vise versa) using TCP Socket.

If yes, can any one give me some sample.


Thanks in advance.

[jhmckeever]

Hi thirupri,

Without wishing to sound rude, your question really doesn't make much sense. <a href="http://en.wikipedia.org/wiki/Extract,_t ... ad">ETL</a> is a process used in Data Warehousing (or other data migration activity) whereas TCP sockets are a network communications mechanism.

As this is a forum related specifically to DataStage and its related tools I'll assume your question is "Does DataStage support inter-process communications using Sockets over TCP?". The answer is that DataStage supports anything you can do on the host operating system using a number of different mechanisms for calling external programs or library functions.

My personal experience of using sockets from DataStage is to write a pair of programs in 'C' - A client (requester) and a server (listener). The client is wrapped as a BuildOp and every row into the BuildOp stage is read from stdin and fired off to the server process using sockets. Every row returned is then written to stdout for use back in DataStageLand (Hey - that gives me a really good idea for a theme park! Anyone want to invest?)

Read the documentation on custom stages. Tell us more specifically what it is you're trying to achieve using sockets.

J

[eostic]

also, JavaPack is another option to BuildOp, but the concept is the same.

[ray.wurlod]

jhmckeever's response is appropriate to parallel jobs, not server jobs.

There is no direct support for sockets in server jobs, but the Sequential File stage can work with named pipes, which comes close in functionality for inter-job connectivity.

[jhmckeever]

Very true - Because the OP was rather vague in its intent I interpreted the question more broadly than referring only Server jobs.

For the record, the example I gave is also used in server jobs by using named pipes (as Ray rightly suggested) and kicking off the client application using a Before-Job ExecSH Routine, using redirection to get it to read and write those named pipes.

J.

[thirupri]

Dear jhmckeever, Ray & Eostic,

Thanks a lot for the Information. Actually, some one asked me is it possible to use ETL for socket connectivity. We like to kick the cloverleaf out and instead we like to use the Data Stage & Tuxedo (unix) instead.

Once again thanks for the useful information. If any one can give more info related to this subject. I am very thankful to them.


Thanks a lot.

[DSguru2B]

You can mark your post as "Resolved" if your questions are answered.