Sparse lookup on Mainframe db2 using Db2 connector
Moderators: chulett, rschirm, roy
Sparse lookup on Mainframe db2 using Db2 connector
Can we do Sparse look-up on Mainframe db2 table. As per my understanding DB2 EE stage in 7.5 does not connect to Mainframe and DB2 API does not support sparse look-up.
Do we have any extra features in v8.1 to do a sparse look-up on Mainframe db2
Do we have any extra features in v8.1 to do a sparse look-up on Mainframe db2
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
We are planning to use this sparse lookup in a DataStage Job that will be exposed as a web-service. The web-service performance should be very high.
These are the two options I am thinking of
1) use ODBC connector and do a sparse look on Mainframe db2.
2) Load Mainframe db2 data into local aix db2 and do sparse lookup on this database using db2 connector.
Please provide your thoughts which option will be the right way.
These are the two options I am thinking of
1) use ODBC connector and do a sparse look on Mainframe db2.
2) Load Mainframe db2 data into local aix db2 and do sparse lookup on this database using db2 connector.
Please provide your thoughts which option will be the right way.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Ray,
The deisgn is for a web service, where we will have one request (one record), which will have to find records in a reference database where we have few million rows.
So I thought Sparse is a best fit here. And the performance I am talking about is firing the query on database and get the results. Will ODBC connector works fine in this scenario.
The deisgn is for a web service, where we will have one request (one record), which will have to find records in a reference database where we have few million rows.
So I thought Sparse is a best fit here. And the performance I am talking about is firing the query on database and get the results. Will ODBC connector works fine in this scenario.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
A few million rows (being selective about the columns - only the ones you need) may well fit in memory and be amenable to normal, local lookup, which is fast. Use an always-on job so that the reference data set remains loaded in memory.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
We are planning to use always on job, which will be accessed by concurrent users. The reference data with minimal number of columns is around 2 GB. And when we expose this job as webservice, we might need minimum 3-4 instances of the service running in ISD for supporting concurrent requests.
Since it is consuming lot of memory, we wanted to go for a sparse lookup. The other point we have analyzed is that the normal local lookup data will be loaded into the memory at the start of the service and is not available for any updates. We will have few real time updates everyday on the reference, which should be availble for the reference making the service search near real time.
We are planning to load Mainframe data to a local aix database table and use db2 connector to do sparse lookup.
Since it is consuming lot of memory, we wanted to go for a sparse lookup. The other point we have analyzed is that the normal local lookup data will be loaded into the memory at the start of the service and is not available for any updates. We will have few real time updates everyday on the reference, which should be availble for the reference making the service search near real time.
We are planning to load Mainframe data to a local aix database table and use db2 connector to do sparse lookup.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: