Replacing Tables with a Stage in Datastage
Posted: Sun Nov 11, 2012 3:45 pm
Hello All,
I am pretty new to Datastage 8.5 and we are working on Sybase database. we are experiencing performance issues since we deal with so much of extraction queries in every jobs. I am wondering is there a way we can use a stage in Datastage as a table and refer to that stage everytime to do all Database operations (Insert, update , delete, select) to avoid lot of I/O with Sybase .
Let me explain it further.
We have a table A which has 10 columns.
there are some History tables - Table B and Table C which act as references for our data processing.
In the first job, we will populate incoming data in to Table A (only 4 columns we will populate)
Based on these 4 column values, we will lookup Table B and populate 5th and 6th column in Table A.
After that based on 4,5 and 6th column in Table A, we will lookup with Refernce Table C and populate the remaining columns in Table A.
Because of this, we are doing lot ot I/O and the process is very slow even though we have tuned our sybase queris.
My question here is , is there any way we can make these tables A, B and C into a stage in Datastage so that we only deal within Datastage server. We don't need to hit the database everytime.
Any thoughts?
I am pretty new to Datastage 8.5 and we are working on Sybase database. we are experiencing performance issues since we deal with so much of extraction queries in every jobs. I am wondering is there a way we can use a stage in Datastage as a table and refer to that stage everytime to do all Database operations (Insert, update , delete, select) to avoid lot of I/O with Sybase .
Let me explain it further.
We have a table A which has 10 columns.
there are some History tables - Table B and Table C which act as references for our data processing.
In the first job, we will populate incoming data in to Table A (only 4 columns we will populate)
Based on these 4 column values, we will lookup Table B and populate 5th and 6th column in Table A.
After that based on 4,5 and 6th column in Table A, we will lookup with Refernce Table C and populate the remaining columns in Table A.
Because of this, we are doing lot ot I/O and the process is very slow even though we have tuned our sybase queris.
My question here is , is there any way we can make these tables A, B and C into a stage in Datastage so that we only deal within Datastage server. We don't need to hit the database everytime.
Any thoughts?