MQ connector stage - composite messages and sequence number

ssunda6 · Post by **ssunda6** » Wed Sep 05, 2012 8:51 am

Hi,

We send MQ messages. The receiving system has limitation on the size of MQ message to receive, so if the message size is huge then we need to break messages and send them.

The file sent contains header(1st record), data records and trailer(last record). If the message has to be broken because of size then we need to still send the messages as belonging to a single group and sequence number should increment for each message.

I have tried segmented message option and also row buffer count property to indicate the number of rows to go into a composite message. But the sequence number is constant for all these messages.

I can before-hand split the file into required subsets and then assign the calculated sequence number. But if the messages are sent as individual messages and because of some error it delivers only few messages then the receiving team will have difficultly knowing if they will receive next messages or not. Please suggest if there is a better solution or any property settings in connector which can help.

eostic · Post by **eostic** » Wed Sep 05, 2012 10:46 am

...been a long time since I've played in detail with the various options on writing to a queue, but my initial gut reaction would be to break up the data on my own, so that I can have explicit control of what is happening, and then feed a column value of my own making into the Correlation ID (for a common value across all the messages from one particular file) and perhaps also the Message ID (for naming them individually).

Worst case, add to the payload with two columns directly in front of each message payload.

The datatypes of those may not be "that" flexible, so it would take a lot of testing and playing around. Sorry I don't have time to try anything, but that would be where I'd start my research.

Ernie

ssunda6 · Post by **ssunda6** » Wed Sep 05, 2012 10:58 am

Thanks for the reply.
But the receiving system will have difficulty - because of some error if header/trailer is not sent then they will not know how many records were supposed to come and also that will be kind of indefinite wait right!

The main challenge I am having is with the sequence number.
In the manuals, it says

"Message sequence number- Use this property to specify the sequential number of the logical message within a message group. Each logical message in a message group is assigned a number beginning with 1 for the first message. This number is increased at a rate of one for each new logical message in this group. Physical messages that are not part of a group are all assigned a sequence number of 1"

But the problem is, it is not incrementing the sequence number.
I tried with segmented message - 'yes/no' both options and with just specifying 'Row buffer count' also. For all logical messages it is using the same sequence number which is specified in the job as default value.

eostic · Post by **eostic** » Wed Sep 05, 2012 3:17 pm

perhaps...but if you craft the solution yourself, you can pass the total number of expected messages for each file in the header....