Search found 39 matches

by raghav_ds
Tue Aug 30, 2011 6:08 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sparse lookup on Mainframe db2 using Db2 connector
Replies: 9
Views: 4765

We are planning to use always on job, which will be accessed by concurrent users. The reference data with minimal number of columns is around 2 GB. And when we expose this job as webservice, we might need minimum 3-4 instances of the service running in ISD for supporting concurrent requests. Since i...
by raghav_ds
Mon Aug 29, 2011 11:12 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sparse lookup on Mainframe db2 using Db2 connector
Replies: 9
Views: 4765

Ray, The deisgn is for a web service, where we will have one request (one record), which will have to find records in a reference database where we have few million rows. So I thought Sparse is a best fit here. And the performance I am talking about is firing the query on database and get the result...
by raghav_ds
Thu Aug 25, 2011 8:44 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sparse lookup on Mainframe db2 using Db2 connector
Replies: 9
Views: 4765

We are planning to use this sparse lookup in a DataStage Job that will be exposed as a web-service. The web-service performance should be very high. These are the two options I am thinking of 1) use ODBC connector and do a sparse look on Mainframe db2. 2) Load Mainframe db2 data into local aix db2 a...
by raghav_ds
Wed Aug 24, 2011 11:52 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Sparse lookup on Mainframe db2 using Db2 connector
Replies: 9
Views: 4765

Sparse lookup on Mainframe db2 using Db2 connector

Can we do Sparse look-up on Mainframe db2 table. As per my understanding DB2 EE stage in 7.5 does not connect to Mainframe and DB2 API does not support sparse look-up.

Do we have any extra features in v8.1 to do a sparse look-up on Mainframe db2
by raghav_ds
Wed Aug 10, 2011 6:56 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Complex Mainframe Parsing
Replies: 1
Views: 2045

Complex Mainframe Parsing

I have a COBOL File coming from mainframe as follows: 1) Contains 60 different record types. 2) Record type can be identified based on a fixed header field 3) Based on header field, identify the record type and we have separate copybook for each record type. eg: For record type 99, the data will be ...
by raghav_ds
Fri May 27, 2011 3:23 am
Forum: IBM QualityStage
Topic: Design for real time reference data update.
Replies: 9
Views: 4959

Ernie, Thank you for your explanation. I will try with database and sparse lookups. SURA, Please find answers for your questions. 1. How much changes will you expect a day? The web service would be used at enterprise level and we are having huge number of users who will be invoking it. 1a. How frequ...
by raghav_ds
Thu May 26, 2011 3:49 am
Forum: IBM QualityStage
Topic: Design for real time reference data update.
Replies: 9
Views: 4959

SURA,
I am not able to see your reply. If you have added some information in the thread, could you please update it again.

Thanks
Raghav
by raghav_ds
Wed May 25, 2011 9:35 pm
Forum: IBM QualityStage
Topic: Design for real time reference data update.
Replies: 9
Views: 4959

Ernie,

Sparse lookup option may not be a good option for us as we need to hit the database for each web-service invocation.

Could you please provide some input on " crafting your own in-memory objects using Java or some other technology".

Thanks
Raghav
by raghav_ds
Wed May 25, 2011 1:15 pm
Forum: IBM QualityStage
Topic: Design for real time reference data update.
Replies: 9
Views: 4959

Design for real time reference data update.

We have created a webservice for customer search (both deterministic and probabilistic matches). The customer will be searched on a reference data of 200 million records. Currently we have designed the webservice for static reference data which needs to be updated for realtime scenario. Current desi...