Here is the scenario:
We have an instance of SalesForce.com that we're wanting to get data from and push data to on a nightly basis. Our current job design consists of the following: An Oracle stage that reads in the columns requested from SFDC, which is passed to an Java Transform stage. In that stage, the jar files are invoked, in which they extract the column names and create a "select" statement to extract the data from SFDC. The java Transform stage then links to another Oracle Stage or a sequential file where the selected data is output.
We have been successful in extracting data from SFDC, but when it comes to larger volumns, we have issues, like the job just hanging. It seems to be related to the number of columns times the number of records returned.
We've tried numerous different scenarios with different number of columns and more rows. We think it may be memory related, but not sure.
First of all.....yes, I know there is a purchasable SFDC adapter for DataStage, but we have written one in Java for the client.
Has anyone else experienced this sort of pain?
![Smile :-)](./images/smilies/icon_smile.gif)
Thanks.