High volume of reference data
Posted: Thu Mar 10, 2011 1:15 am
Hi All,
We are getting high volume of reference data is one of my job and this job is taking 40 min time for complete execution.
On daily basis they are adding reference data and at this point of time we are getting more than 50 million records.
We have tried few options but the issue is still same.
1. tried with join and merge instead of look up
2. tried with recordset dataset as reference data instead of refering against the table directly
3. tried with creating indexes on the table
And also there is no limit for this reference data there is a chance of getting more volume in future.
Anyone please suggest if you have any idea on this.
Thanks
We are getting high volume of reference data is one of my job and this job is taking 40 min time for complete execution.
On daily basis they are adding reference data and at this point of time we are getting more than 50 million records.
We have tried few options but the issue is still same.
1. tried with join and merge instead of look up
2. tried with recordset dataset as reference data instead of refering against the table directly
3. tried with creating indexes on the table
And also there is no limit for this reference data there is a chance of getting more volume in future.
Anyone please suggest if you have any idea on this.
Thanks