Job aborted 'glibc detected' error

[rohitagarwal15]

One of my job is getting aborted giving error *** glibc detected *** but rest all other jobs are running fine..
In my job i am loading data in db2 table but when i run this job for single record and put dataset in target then it runs fine..
I search the forum also but didn't find the resolution for this issue..
My OS is LINUX

[chulett]

Post the entire, unedited error message or messages.

[mhester]

Would this job design incorporate a stored procedure stage?

What version of the glibc etc... are on your system?

[rohitagarwal15]

No we are not using any stored procedure..
Can u tell me what do you mean by "version of glibc"
Today again when i run the job for new table it is aborted with the same error as lot of warnings came so i have not posted all warnings but some of them only..
Pls provide your valuable suggestions..

db2_api_insert_tertiary_sales,0: *** glibc detected *** /opt/IBM/InformationServer/Server/PXEngine/bin/osh: malloc(): memory corruption (fast): 0x0952ae58
db2_api_insert_tertiary_sales,0: /lib/libc.so.6(__libc_start_main+0xdc)[0xa95e9c]
db2_api_insert_tertiary_sales,0: ======= Memory map: ========
db2_api_insert_tertiary_sales,0: /opt/IBM/InformationServer/Server/PXEngine/bin/osh(__gxx_personality_v0+0xb1)[0x804bb45]
db2_api_insert_tertiary_sales,0: /lib/libc.so.6[0xaeac76]
db2_api_insert_tertiary_sales,0: Operator terminated abnormally: received signal SIGABRT

[chulett]

malloc(): memory corruption (fast) ? In your shoes, I be getting that in front of my official support provider.

[mhester]

No we are not using any stored procedure..
Can u tell me what do you mean by "version of glibc"

In many cases IBM has suggested that reverting to an older version of the glibc libraries can fix this problem. It does not necessarily happen with all job runs and with all types of jobs. You may even have success with this particular job and then sporadically experience a failure.

On your system you can verify which glibc libraries are present and installed by performing the following command from Unix -

Code: Select all

rpm -qa | grep glibc

I suspect that support will first have you define a new user environment variable named -

malloc_check_ (yes there is an underscore after the check) and set its value to False or 0 and then run the job. This should only be set at the job level to False and not at the project level. If this takes care of the problem then you can probably leave it as is and continue. If it does not then opening a support ticket with IBM would be in order and they can most certainly walk you through how to determine which libraries may/may not need to be reverted or updated.

If this produces a core then they will likely want that also. If the process is long running or you have time before it aborts you could always attach a debugger to that operators pid (turn on APT_PM_SHOW_PIDS). By doing this you will understand a bit more about what is happening.

[mhester]

On a related note - I would definitely make sure that all of your columns have defined lengths. I know that there is talk that the framework is more efficient if you do not define lengths, but that is only true if the length is greater than 100. Anything less than 100 should be defined especially dates and timestamps.

The following is from the development product manager (in charge of DataStage) at IBM.

We recommend defining a length unless the max length is large (over ~100 bytes) and seldom used, as the record processing layer will be more efficient if a length is specified.
There was a bit of a time/space trade-off, as datasets used to be created with the added padding, but we have since added an env. var (which is the default in 8.5) so when writing these out, we remove the padding.

[rohitagarwal15]

When er execute the command rpm -qa | grep glibc following is the output.

compat-glibc-2.3.4-2.26
glibc-common-2.5-49.el5_5.7
compat-glibc-headers-2.3.4-2.26
compat-glibc-2.3.4-2.26
glibc-devel-2.5-49.el5_5.7
glibc-2.5-49.el5_5.7
glibc-devel-2.5-49.el5_5.7
glibc-headers-2.5-49.el5_5.7
glibc-2.5-49.el5_5.7

Also we have checked all of the columns have defined lengths..

[pandeesh]

When er execute the command rpm -qa | grep glibc following is the output.

May i know where we need to execute the command rpm -qa|grep glibc

It's not found on my unix version.
glibc libraries are found only in 8x or can we find them in 7.5.x too?

Thanks

[ray.wurlod]

rpm is a Linux command. It may not be findable in your PATH, but it should be findable if you're logged as superuser. Of course, you could use a find command to determine the location of the rpm command.

[mhester]

When er execute the command rpm -qa | grep glibc following is the output.

compat-glibc-2.3.4-2.26
glibc-common-2.5-49.el5_5.7
compat-glibc-headers-2.3.4-2.26
compat-glibc-2.3.4-2.26
glibc-devel-2.5-49.el5_5.7
glibc-2.5-49.el5_5.7
glibc-devel-2.5-49.el5_5.7
glibc-headers-2.5-49.el5_5.7
glibc-2.5-49.el5_5.7

Also we have checked all of the columns have defined lengths..

I cannot ascertain from this listing which libraries might need to be updated/reverted, if any. That is really a call for support. I would suggest comparing these versions to what is called for in your installation documents for the toolset.

[way246]

Resolving the problem
Set environment variable MALLOC_CHECK_=0

https://www-304.ibm.com/support/docview ... wg21410927