XML field truncation

[joffijoy]

Hi,

I have a Job that read XML from a MQ. Size of the XML field is defined as varchar(999999). But I am getting the XML truncated after 100002 Bytes.

Job reads the XML from MQ and writes to a file.

The environment variables I have are:

APT_AUTO_TRANSPORT_BLOCK_SIZE = False
APT_DEFAULT_TRANSPORT_BLOCK_SIZE = 400000
APT_MAX_TRANSPORT_BLOCK_SIZE = 1048576
APT_LATENCY_COEFFICIENT = 5

The XMLs can go up to 6 GB.

What should I do to prevent truncation.

Thank you,
Joffi

[eostic]

...hard to tell exactly what might be causing the issue, but two things you should look at:

the max size supported for your MQ queue. Do you have another application capable of browsing the queue? It may already be truncated if the MQ queue is defined too small.

do a test using a Server job......just to see if it's an APT setting problem.

also --- beware that about the largest document you'll be able to process, in general, is 500M.

Ernie

[joffijoy]

Hi,

I have a Job that read XML from a MQ. Size of the XML field is defined as varchar(999999). But I am getting the XML truncated after 100002 Bytes.

Job reads the XML from MQ and writes to a file.

The environment variables I have are:

APT_AUTO_TRANSPORT_BLOCK_SIZE = False
APT_DEFAULT_TRANSPORT_BLOCK_SIZE = 400000
APT_MAX_TRANSPORT_BLOCK_SIZE = 1048576
APT_LATENCY_COEFFICIENT = 5

The XMLs can go up to 6 GB.

What should I do to prevent truncation.

Thank you,
Joffi

Solved the Issue!. It turned out that while defining the column for MQ stage I had less number of bytes that Column size filed. Hard to belive!, but once i changed the display field as that of column size, problem solved.

[chulett]

Well then, you need to mark your post as Resolved as well.

[sud]

Solved the Issue!. It turned out that while defining the column for MQ stage I had less number of bytes that Column size filed. Hard to belive!, but once i changed the display field as that of column size, problem solved.

Interesting, so was the data ACTUALLY getting truncated or was it just a view data problem? How were you checking the data?

[eostic]

Now that I think about it, I'm fairly certain I recall that the plugin was designed to truncate (as opposed to aborting) if the column(s) were too small, and pad if the column is too big. There were lots of situations early on where sites didn't have all the meta data for their queues or had variable messages with dramatic changes in size from msg to msg....Doing trial and error, while not easy in any situation, is far more difficult if the software complains and dies. So it was decided that it would be more flexible, truncating in case of too short and padding if too long. Too bad it didn't issue a warning while doing so.

Ernie