Datastage General Questions

[harryhome]

1. Transaction Management. Are transactions always on a stage level? Is it possible for a transaction to span multiple stages and/or jobs?

2. Database connection/session handling. Is there connection pooling in Datastage? If yes, can concurrent transactions be prevented in a single db session?

3. Oracle connector. In some cases Oracle connector appends 'returning' clause to the sql statement (update). Is this controllable?

4. Oracle connector. Does Datastage support Oracle record types? Can we specify row%type, column%type etc.?

5. Messages availability/propagation. What is the best practice to pass messages/records between jobs? (persistent data sets?)

6. Logging. What is the best practice to log a message/record when it traverses various stages?
Does DataStage have a config parameter for implicit record logging or it has to be explicitly done in the job?

[ray.wurlod]

1. Each stage (conceptually at least) becomes a separate process so that there is, in general, no way for a transaction to span multiple stages. However, the Distributed Transaction Stage does allow a transaction to span multiple links.

2. There is no configurable connection pooling in DataStage. If the database client software implements connection pooling and this is enabled in the client configuration then it may be used, but this is not controlled by DataStage.

3. I don't believe that it's controllable, but would be happy to be corrected if someone knows how it can be controlled. (I have not used this Connector much.)

4. Same as answer #3.

5. Persistent data sets for data. Files or queues for messages.

6. "All of the above". There are default logging levels, and environment variables that can be set to get additional information logged. Beyond that, you can compile in trace mode which, among other things, adds a Peek operator to every link in the job design.