DSXchange

Hello Forum,

I don't have so much experience with Real time ETL products. My understanding is that MQ Series is usually used in real time processing to consume messages produced a registered source.

So my questions are:

1. Does the MQ Stage perform any real time functionality. Or does it just read messages in batch when I explicitly schedule my job to run?

2. Is the MQ stage suppose/has to be run with RTI/Soa package?

3. If I do use MQ stage within the Soa package, do I get any benefit?

Your input is appreciated.

Greg

This is a very old entry, but this question comes up all the time, so I figured it was worth addressing.

The MQ Series stage "listens" for new messages on a message queue, which means that the job (by default) will run forever, "waiting" for new messages to arrive. There are, of course, elegant options to terminate the job (after a certain amount of time has elapsed, a number of messages, or a special MQ Message Type is received). It's advised that you always have a way to stop the job from running.

The MQ Stage is entirely separate from RTI..... RTI is more generic, also allowing jobs to be "always on," but with a lot more infrastructure to support message queues, SOAP input, direct Java (EJB) invocation, and things like load balancing and failover.

They are certainly similar, but with MQ it's "just a stage," so you have to be responsible for all the load balancing and environment set-up, if desired.

Additionally, with MQ you have to be careful about using an "always on" job --- and watch out for blocking stages such as the Aggregator. With RTI, DataStage understands when a single "message" is complete, allowing you to utilize the Aggregator and other such stages.

Ernie

eostic wrote:With RTI, DataStage understands when a single "message" is complete, allowing you to utilize the Aggregator and other such stages.

Interesting, not something I was really aware of. I was under the impression that stages that do their dirty work when the input stream ends could be... less than functional in an RTI job. Somewhat like the OCI issue you helped us work out when Julio was here.

Thanks for the info, Ernie! Have to build some test cases and play when I have some spare time.