Is There A Limit To Number Of Columns In Resultset

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Is There A Limit To Number Of Columns In Resultset

Post by admin »

I have a job which uses a view that returns 90 columns in the resultset (we have our reasons). This resultset is then used to populate a table. When I run this job, it aborts with the following:

Program "DSD.BCIOpenW": Line 437, Array index out of bounds. Attempting to Cleanup after ABORT raised in stage DWCustomer.DMMatView DataStage Phantom Aborting with @ABORT.CODE = 3

I noticed as long as I have up to 82 columns, the job runs fine. As soon as I add a 83rd column, regardless of which column, I get the above error. I can View Data from the stage no problems, even with all 90 columns. This is with ODBC stages on DataStage 3.6.2 (Tru64 unix).

Anyone encountered this before?

Regards
Hong
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hong,

We have tables with significantly more than 90 columns. (I wont reveal how many, it is too embarrassing.) However, these tables are updated using the ORAOCI8 stage. I dont believe we have attempted this with ODBC stages.

I guess this probably does not help.

David Barham
Information Technology Consultant
CoalMIS Project
Anglo Coal Australia Pty Ltd
Brisbane, Australia

-----Original Message-----
From: Hong Kuan [SMTP:HongK@bsbs.co.nz]
Sent: Friday, 18 August 2000 11:39
To: informix-datastage@oliver.com
Subject: Is There A Limit To Number Of Columns In Resultset

I have a job which uses a view that returns 90 columns in the resultset (we
have our reasons). This resultset is then used to populate a table. When I
run this job, it aborts with the following:

Program "DSD.BCIOpenW": Line 437, Array index out of bounds.
Attempting to Cleanup after ABORT raised in stage DWCustomer.DMMatView
DataStage Phantom Aborting with @ABORT.CODE = 3

I noticed as long as I have up to 82 columns, the job runs fine. As soon as
I add a 83rd column, regardless of which column, I get the above error. I
can View Data from the stage no problems, even with all 90 columns. This is
with ODBC stages on DataStage 3.6.2 (Tru64 unix).

Anyone encountered this before?

Regards
Hong




*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else

*************************************************************************
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

There are two restrictions.

By default, the maximum number of columns is 400. The maximum number of bytes in a row in the result set is 8192. You may be encountering this limit.

The good news is that these are not hard limits. Add the following lines to the bottom of the uvodbc.config file in your project directory to change them. Note that the spaces around the equals signs are compulsory.

MAXFETCHCOLS = 666
MAXFETCHBUFF = 16384

These values will allow a maximum of 666 columns and 16K bytes per row.

Hope this helps.

> ----------
> From: David Barham[SMTP:David.Barham@anglocoal.com.au]
> Reply To: informix-datastage@oliver.com
> Sent: Friday, 18 August 2000 12:33
> To: informix-datastage@oliver.com
> Subject: RE: Is There A Limit To Number Of Columns In Resultset
>
> Hong,
>
> We have tables with significantly more than 90 columns. (I wont
> reveal how many, it is too embarrassing.) However, these tables are
> updated using the
> ORAOCI8 stage. I dont believe we have attempted this with ODBC stages.
>
> I guess this probably does not help.
>
> David Barham
> Information Technology Consultant
> CoalMIS Project
> Anglo Coal Australia Pty Ltd
> Brisbane, Australia
>
> -----Original Message-----
> From: Hong Kuan [SMTP:HongK@bsbs.co.nz]
> Sent: Friday, 18 August 2000 11:39
> To: informix-datastage@oliver.com
> Subject: Is There A Limit To Number Of Columns In Resultset
>
> I have a job which uses a view that returns 90 columns in the
> resultset (we
> have our reasons). This resultset is then used to populate a table.
> When I
> run this job, it aborts with the following:
>
> Program "DSD.BCIOpenW": Line 437, Array index out of bounds.
> Attempting to Cleanup after ABORT raised in stage
> DWCustomer.DMMatView
> DataStage Phantom Aborting with @ABORT.CODE = 3
>
> I noticed as long as I have up to 82 columns, the job runs fine. As
> soon as
> I add a 83rd column, regardless of which column, I get the above
> error. I
> can View Data from the stage no problems, even with all 90 columns.
> This is
> with ODBC stages on DataStage 3.6.2 (Tru64 unix).
>
> Anyone encountered this before?
>
> Regards
> Hong
>
>
>
>
> **********************************************************************
> ***
> This e-mail and any files transmitted with it may be confidential
> and are intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this e-mail in
> error, please notify the sender by return e-mail, and delete this
> e-mail from your in-box. Do not copy it to anybody else
>
> **********************************************************************
> ***
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hong,

I run a DS job that inserts/updates an Oracle table with more than 90 columns using the Merant ODBC driver with no problems. What DBMS are you trying to connect to using ODBC? I was just curious. Ive had ugly hacks with Informix ODBC though related to other problems. However, reading the error message with references to "array index" and "matView" makes me speculate this may come from the underlying Universe system but, Im not really sure. If you have Ardent (Informix) support then give them a call. They may have an answer.

Mark Detwiler
Detwiler Consulting
mailto:mark@msdetwiler.com



-----Original Message-----
From: Hong Kuan
To: informix-datastage@oliver.com
Date: Friday, August 18, 2000 9:37 PM
Subject: Is There A Limit To Number Of Columns In Resultset


>I have a job which uses a view that returns 90 columns in the resultset
>(we have our reasons). This resultset is then used to populate a table.
>When I run this job, it aborts with the following:
>
>Program "DSD.BCIOpenW": Line 437, Array index out of bounds. Attempting
>to Cleanup after ABORT raised in stage DWCustomer.DMMatView DataStage
>Phantom Aborting with @ABORT.CODE = 3
>
>I noticed as long as I have up to 82 columns, the job runs fine. As
>soon as I add a 83rd column, regardless of which column, I get the
>above error. I can View Data from the stage no problems, even with all
>90 columns. This is with ODBC stages on DataStage 3.6.2 (Tru64 unix).
>
>Anyone encountered this before?
>
>Regards
>Hong
>
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hi Ray

Thanks for the suggestion. The view/table were using is well within the defaults, at 91 columns and 1271 bytes per row. I went ahead and added the two max settings to our uvodbc.config in $uvhome. Same error still occurring. Must be something else.

Ive since placed a call with tech support. See what they to say.

Thanks again.
Hong

> -----Original Message-----
> From: Ray Wurlod [SMTP:ray.wurlod@informix.com]
> Sent: Friday, August 18, 2000 5:42 PM
> To: informix-datastage@oliver.com
> Subject: RE: Is There A Limit To Number Of Columns In Resultset
>
> There are two restrictions.
>
> By default, the maximum number of columns is 400. The maximum number
> of bytes in a row in the result set is 8192. You may be encountering
> this limit.
>
> The good news is that these are not hard limits. Add the following
> lines to the bottom of the uvodbc.config file in your project
> directory to change them. Note that the spaces around the equals
> signs are compulsory.
>
> MAXFETCHCOLS = 666
> MAXFETCHBUFF = 16384
>
> These values will allow a maximum of 666 columns and 16K bytes per
> row.
>
> Hope this helps.
>
> > ----------
> > From: David Barham[SMTP:David.Barham@anglocoal.com.au]
> > Reply To: informix-datastage@oliver.com
> > Sent: Friday, 18 August 2000 12:33
> > To: informix-datastage@oliver.com
> > Subject: RE: Is There A Limit To Number Of Columns In Resultset
> >
> > Hong,
> >
> > We have tables with significantly more than 90 columns. (I wont
> > reveal how many, it is too embarrassing.) However, these tables are
> > updated using the
> > ORAOCI8 stage. I dont believe we have attempted this with ODBC stages.
> >
> > I guess this probably does not help.
> >
> > David Barham
> > Information Technology Consultant
> > CoalMIS Project
> > Anglo Coal Australia Pty Ltd
> > Brisbane, Australia
> >
> > -----Original Message-----
> > From: Hong Kuan [SMTP:HongK@bsbs.co.nz]
> > Sent: Friday, 18 August 2000 11:39
> > To: informix-datastage@oliver.com
> > Subject: Is There A Limit To Number Of Columns In Resultset
> >
> > I have a job which uses a view that returns 90 columns in the
> > resultset (we
> > have our reasons). This resultset is then used to populate a table.
> > When I
> > run this job, it aborts with the following:
> >
> > Program "DSD.BCIOpenW": Line 437, Array index out of bounds.
> > Attempting to Cleanup after ABORT raised in stage
> > DWCustomer.DMMatView
> > DataStage Phantom Aborting with @ABORT.CODE = 3
> >
> > I noticed as long as I have up to 82 columns, the job runs fine. As
> > soon as
> > I add a 83rd column, regardless of which column, I get the above
> > error. I
> > can View Data from the stage no problems, even with all 90 columns.
> > This is
> > with ODBC stages on DataStage 3.6.2 (Tru64 unix).
> >
> > Anyone encountered this before?
> >
> > Regards
> > Hong
> >
> >
> >
> >
> >
> **********************************************************************
> ***
> > This e-mail and any files transmitted with it may be confidential
> > and are intended solely for the use of the individual or entity to
> > whom they are addressed. If you have received this e-mail in error,
> > please notify the sender by return e-mail, and delete this e-mail
> > from your in-box. Do not copy it to anybody else
> >
> >
> **********************************************************************
> ***
> >
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Hi Mark

We are using Merants ODBC driver for Informix, the version that came with DS 3.6.3. Im curious to what hacks you needed with Informix ODBC. Can you share your experince with us? We are not statisfied with the driver ourselves on Tru64 Unix. In particular the performance. Not to mention the "nagware" (a warning mesage after every successful job - Merants KB implies the driver is not licensed), unable to import table definitions, error messages coming back as numbers only, etc. We have stuck with ODBC because the alternative, CLI, did not support some features we needed right away.

On the performance side, the same job running on NT (PII 200 with 160MB RAM) against the same DB server is approx. 3 times faster than that on Tru64 (DEC Alpha 333Mhz, 1 CPU, 1GB RAM). This doesnt make sense! We have gone back to Informix to help resolve this issue. But, if you or anyone else have any suggestions, it most welcome.

Regards
Hong

> -----Original Message-----
> From: Mark Detwiler [SMTP:mark@msdetwiler.com]
> Sent: Saturday, August 19, 2000 6:06 PM
> To: informix-datastage@oliver.com
> Subject: Re: Is There A Limit To Number Of Columns In Resultset
>
> Hong,
>
> I run a DS job that inserts/updates an Oracle table with more than 90
> columns using the Merant ODBC driver with no problems. What DBMS are
> you trying to connect to using ODBC? I was just curious. Ive had
> ugly hacks with Informix ODBC though related to other problems.
> However, reading the error message with references to "array index"
> and "matView" makes me speculate this may come from the underlying
> Universe system but, Im not really sure. If you have Ardent
> (Informix) support then give them a call. They may have an answer.
>
> Mark Detwiler
> Detwiler Consulting
> mailto:mark@msdetwiler.com
>
>
>
> -----Original Message-----
> From: Hong Kuan
> To: informix-datastage@oliver.com
> Date: Friday, August 18, 2000 9:37 PM
> Subject: Is There A Limit To Number Of Columns In Resultset
>
>
> >I have a job which uses a view that returns 90 columns in the
> >resultset
> (we
> >have our reasons). This resultset is then used to populate a table.
> >When
> I
> >run this job, it aborts with the following:
> >
> >Program "DSD.BCIOpenW": Line 437, Array index out of bounds.
> >Attempting to Cleanup after ABORT raised in stage
> >DWCustomer.DMMatView DataStage Phantom Aborting with @ABORT.CODE = 3
> >
> >I noticed as long as I have up to 82 columns, the job runs fine. As
> >soon
> as
> >I add a 83rd column, regardless of which column, I get the above
> >error. I can View Data from the stage no problems, even with all 90
> >columns. This
> is
> >with ODBC stages on DataStage 3.6.2 (Tru64 unix).
> >
> >Anyone encountered this before?
> >
> >Regards
> >Hong
> >
> >
>
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

I wish I could be of more help, but the ODBC is the slowest option there is. What you might try is writing a sql script that pulls the data out of the DB natively and creates a flat file then have the DS job use the flat file as input. You should see a huge improvement there. As far as error messages go, we get one for ever job we run and it is caused by a bug that is supposed to be fixed in 4.0.

Good Luck

Mark Griffin
AMP Sr. Application Developer
Data Warehouse Systems
CIGNA Healthcare
Two College Park Drive, 454
Hooksett, NH 03106
Phone: 603-430-7610
603-268-7239
Fax: 603-268-7909
email: griffimk@hlthsrc.com

>>> HongK@bsbs.co.nz 08/22 8:03 PM >>>
Hi Mark

We are using Merants ODBC driver for Informix, the version that came with DS 3.6.3. Im curious to what hacks you needed with Informix ODBC. Can you share your experince with us? We are not statisfied with the driver ourselves on Tru64 Unix. In particular the performance. Not to mention the "nagware" (a warning mesage after every successful job - Merants KB implies the driver is not licensed), unable to import table definitions, error messages coming back as numbers only, etc. We have stuck with ODBC because the alternative, CLI, did not support some features we needed right away.

On the performance side, the same job running on NT (PII 200 with 160MB RAM) against the same DB server is approx. 3 times faster than that on Tru64 (DEC Alpha 333Mhz, 1 CPU, 1GB RAM). This doesnt make sense! We have gone back to Informix to help resolve this issue. But, if you or anyone else have any suggestions, it most welcome.

Regards
Hong

> -----Original Message-----
> From: Mark Detwiler [SMTP:mark@msdetwiler.com]
> Sent: Saturday, August 19, 2000 6:06 PM
> To: informix-datastage@oliver.com
> Subject: Re: Is There A Limit To Number Of Columns In Resultset
>
> Hong,
>
> I run a DS job that inserts/updates an Oracle table with more than 90
> columns using the Merant ODBC driver with no problems. What DBMS are
> you trying to connect to using ODBC? I was just curious. Ive had
> ugly hacks with Informix ODBC though related to other problems.
> However, reading the error message with references to "array index"
> and "matView" makes me speculate this may come from the underlying
> Universe system but, Im not really sure. If you have Ardent
> (Informix) support then give them a call. They may have an answer.
>
> Mark Detwiler
> Detwiler Consulting
> mailto:mark@msdetwiler.com
>
>
>
> -----Original Message-----
> From: Hong Kuan
> To: informix-datastage@oliver.com
> Date: Friday, August 18, 2000 9:37 PM
> Subject: Is There A Limit To Number Of Columns In Resultset
>
>
> >I have a job which uses a view that returns 90 columns in the
> >resultset
> (we
> >have our reasons). This resultset is then used to populate a table.
> >When
> I
> >run this job, it aborts with the following:
> >
> >Program "DSD.BCIOpenW": Line 437, Array index out of bounds.
> >Attempting to Cleanup after ABORT raised in stage
> >DWCustomer.DMMatView DataStage Phantom Aborting with @ABORT.CODE = 3
> >
> >I noticed as long as I have up to 82 columns, the job runs fine. As
> >soon
> as
> >I add a 83rd column, regardless of which column, I get the above
> >error. I can View Data from the stage no problems, even with all 90
> >columns. This
> is
> >with ODBC stages on DataStage 3.6.2 (Tru64 unix).
> >
> >Anyone encountered this before?
> >
> >Regards
> >Hong
> >
> >
>




------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE: If you have received this e-mail in error, please immediately notify the sender by e-mail at the address shown. This e-mail transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Locked