A process is started executing uv (ultimately uvsh) running DSD.RUN, which is a DataStage BASIC routine that runs server jobs. This obtains job parameter values, or run-time default values for them.
Before starting anything else, the job will execute its before-job subroutine.
In your design, the job process forks a child uv process running DSD.StageRun, which is a DataStage BASIC routine that runs the Transformer stage.
Source code for DSD.RUN and DSD.StageRun are intellectual property of Ascential Software Corporation, and not in the public domain.
The source code for the subroutine that IS the Transformer stage can be viewed in the RT_BPnnn directory, where nnn is the job number.
If all the pieces are in place, a Transformer stage does the following.
Code: Select all
1. Execute a before-stage subroutine.
2. Open all its input and output links (which means that the "Open" function exposed by the passive stage on the other end of that link is executed; for example a file is opened, or a connection established to a database and SQL statement(s) prepared).
3. Initialize all stage variables, in the order in which they are declared.
4. For each row on the stream input link (updating status information for each 1000 rows processed):
(a) get the row from the stream input link (which means that the "Get Next" function exposed by the passive stage is executed and may return a row, an error, or a "no more data" token)
(b) for each reference input link, in the declared execution order:
(i) evaluate reference key expression(s)
(ii) issue a reference input query (which means that the "Get Row By Key" function exposed by the passive stage is executed and may return a row containing data, or a row containing all NULL columns or, in an ODBC or UV stage, multiple rows)
(c) evaluate all stage variables, in the order in which they are declared
(d) initialize the REJECTED variable to "true" (this detects whether a row has been processed on to an output link)
(e) for each output link, in the declared execution order, evaluate its constraint expression (a "rejects" output link has an effective constraint of (REJECTED = @TRUE)) then, if the constraint expression is satisfied, set the REJECTED variable to "false", evaluate each of the column derivation expressions in the order in which they have been declared, then send the row to the output link (which means that the "Put" function exposed by the passive stage is executed, and may return a success code or an error code)
4. Close all output and input links.
5. Execute after-stage subroutine.
When the DSD.StageRun process closes, it notifies its parent. An entry containing the pid and the token "[Done]" appears in the parent process's output file in the &PH& directory.
Any abort messages are captured into the stage's output file in the &PH& directory. If the job has not aborted immediately, these messages are copied from there to the job log. Otherwise, this can be done when the job is reset.
The parent job periodically checks that the child process is still present, by sending it a null signal (kill -0) to the child process.
Once all its active stages have closed, the job process executes its after-job subroutine before exiting.
Both the job process and the stage process periodically update respective entries in the RT_STATUSnnn table in the repository. It is this table that is viewed in the Status and Monitor views in Director, and if performance statistics are being shown on the Designer canvas.
As necessary, entries are written to the RT_LOGnn table in the Repository. This is the table viewed in the Log view in Director.
More complex job designs obtain dependency and other information from the RT_CONFIGnnn table in the Repository.
For the ODBC stage:
- the "Open" function is BCI.Open, which - among other things - issues SQLConnect ,SQLPrepare and SQLExecute calls
the "Get Next" function is BCI.GetNext, which - among other things, issues SQLFetch calls
the "Get Row By Key" function is BCI.GetByKey, which also issues SQLFetch calls
the "Put" function is BCI.Put, which issues SQLExecute calls
the "Close" function is BCI.Close, which - among other things, issues a SQLDisconnect call
Why did you want to know?
How do you plan to use your knowledge?
You can't change the way it works.
If you enable stage tracing for the Transformer stage, and enable all the choices, you can see all this happening. I omitted parameter substitution, determination of property values and other substitutions, for clarity.
If you want a detailed reply, you might be better off posting on ADN.
Why go there when you can get the Full Wurlod right here?