DSX stage names to meaningful stage names

Cranie · Post by **Cranie** » Tue Apr 15, 2008 8:45 am

I have gone through a project export and have a list of the stage names used by the various jobs.

I was wondering if there is a list of DSX stage name to the more meaningful name used in DS to save me looking this up?

i.e. CType is this a server or PX stage and what is its real name?

some are obvious:
1 PxSort
1 sort

Thanks for any info.

chulett · Post by **chulett** » Tue Apr 15, 2008 9:28 am

You would tell what's what from the OLETYPE from what I recall. Somewhere out there I believe that someone (Kim?) posted all of them, perhaps a search would turn them up.

ray.wurlod · Post by **ray.wurlod** » Tue Apr 15, 2008 3:34 pm

CType or OLETYPE is the name of the class from which the object (stage, link, table definition, etc.) was instantiated. Try this query to get a list of job object class names.

Code: Select all

SELECT DISTINCT OLETYPE FROM DS_JOBOBJECTS;

These are NOT stage names. Stage names are allocated automatically as stages are emplaced on the design area (e.g. Transformer12) and changed by the diligent developer immediately thereafter to something more meaningful.

jhmckeever · Post by **jhmckeever** » Tue Apr 15, 2008 7:01 pm

Here's a list of OLETypes I took from our project, along with their corresponding descriptions.

The first column is the OLEType with the description after the semicolon. Ignore the "= **" bit. On our project we use a server routine to verify consistent stage naming across all jobs. This is configuration file for that routine.

Code: Select all

[SEQUENCE]
CCondition				=   	; Branch the execution of a sequence depending on a user defined condition
CEndLoopActivity		=   	; Sequence EndLoop
CExceptionHandler     =   	; Capture general job exceptions
CExecCommandActivity  =   	; Execute an operating system command
CJobActivity			 =   	; Execute a DataStage job
CNotificationActivity =   	; Send an email notification based on user entered criterion
CRoutineActivity	   =  	 ; Execute an existing routine
CSequencer			   =   	; Synchronize the control flow of multiple activities in a job sequence
CStartLoopActivity	 =   	; Sequence StartLoop
CTerminatorActivity	=   	; Sequence Terminator
CUserVarsActivity	  =   	; Sequence UserVariables
CWaitForFileActivity  =   	; Wait for the specified file to appear

[SERVER]
AGGREGATOR			= **		; Form groups or perform other aggregate functions
BCPLOAD				= **		; Sybase & SQL Server bulk loader
CCollector			= **		; Merge up to 64 links into single output link
CFF					 = **		; Complex Flat File stage.
CHashedFileStage	= **		; Typically a keyed reference file
CInterProcess		= **		; Communication channel between stages
CInternal			 = **		; Internal Stage
CODBCStage			= **		; ODBC data access
CPartitioner		 = **		; Partition data out to multiple links
CSeqFileStage		= **		; Sequential data access
CUDT6Stage			= **		; UniData 6 data access
CUniDataStage		= **		; UniData data access
CUvStage			  = **		; UniVerse data access
DRS					 = **		; DRS stage (dynamic RDBMS)
DSDB2				  = **		; IBM DB2 UDB data access
DstxMap				= **		; DataStage TX Map Stage
Folder				 = **		; Read or write data as files in a directory
INFBLK10			  = **		; Informix Bulk Loader
IQBulk12			  = **		; Sybase IQ 12 Bulk Loader
InfmxCLI			  = **		; Informix data access
MERGE				  = **		; Merge data access
ORABULK				= **		; Bulk loader
ORAOCI9				= **		; Oracle OCI stage version 1.1
ORAOCIBL			  = **		; Oracle 8.1 Bulk Loader
Pivot				  = **		; Pivot Plugin version 1.0.0
RowMerger			 = **		; Row Merger Active Plug-in Stage
RowSplitter		  = **		; Row Splitter Active Plug-in Stage
STP					 = **		; The Stored Procedure stage.
SYBASEOC			  = **		; Sybase data access
TDMLoad				= **		; Teradata MultiLoad/TPump/FastExport Stage.
Terabulk			  = **		; Teradata Bulk Loader.
Teradata			  = **		; Teradata data access
UDBLoad				= **		; IBM DB2 UDB Bulk Loader
XMLInput		     = **		; XML Input Stage
XMLOutput			 = **		; XML Output Stage
XMLTransformer	  = **		; XML Transformer Stage
XPSLoad				= **		; Informix XPS Bulk Loader
ftp					 = **		; FTP data access
rdbload				= **		; Redbrick Bulk Loader
sort				   = **		; Sort

[PARALLEL]
Link					     = lk		; Link
Container               = sc		; Container
CTransformerStage       = tx		; Enter data transformations
DRSPX                   = dr		; DRS stage (dynamic RDBMS)
DSDB2PX                 = **		; IBM DB2 UDB data access
DstxMapPX               = **		; DataStage TX Map Stage
INFBLK10PX              = **		; Informix Bulk Loader
IQBulk12PX              = **		; Sybase IQ 12 Bulk Loader
InfmxCLIPX              = **		; Informix data access
ORAOCIBLPX              = ob		; Oracle 8.1 Bulk Loader
PivotPX                 = pv		; Pivot Plugin version 1.0.0
PxAggregator            = ag		; Form groups or perform other aggregate functions
PxCFF                   = cf		; Complex flat file access
PxChangeApply           = ca		; Apply change operations to "before" data set
PxChangeCapture         = cc		; Capture changes between "before" and "after" data sets
PxColumnExport          = ce		; Combine columns into a single binary or string column
PxColumnGenerator       = cg		; Create new columns to add to incoming rows
PxColumnImport          = ci		; Parse apart a single binary or string column into multiple columns
PxCombineRecords        = cr		; Combine records with identical keys into single record
PxCompare               = cm		; Compares two pre-sorted input data sets
PxCompress              = co		; Compress data set to binary stream
PxCopy                  = cp		; Copy input to zero or more outputs
PxDB2                   = **		; DB2/UDB data access
PxDataSet               = ds		; DataStage persistent data set access
PxDecode                = dc		; Decode binary data stream using an operating system command
PxDifference            = di		; Compare "before" and "after" data sets
PxEncode                = en		; Encode data stream using operating system command
PxExpand                = ex		; Expand binary stream to data set
PxExternalFilter        = ef		; Call an operating system command as a filter
PxExternalSource        = es		; Call a program that acts as a data source
PxExternalTarget        = et		; Call a program that acts as a data target
PxFTP                   = ft		; FTP file access
PxFileSet               = fs		; File Set access
PxFilter                = fi		; Assign input rows to output links based on constraint
PxFunnel                = fn		; Combine multiple inputs to single output
PxGeneric               = ge		; Call a user written operator
PxHead                  = hd		; Copy specified number of input rows to output
PxInformixXPS           = **		; Informix data access
PxJoin                  = jn		; Join two inputs (inner, outer or full join) to create one output
PxLookup                = lu		; Lookup on one or more sources or one (sparse) lookup to a direct DB source
PxLookupFileSet         = lf		; File Set used as lookup or reference
PxMakeSubRec            = ms		; Convert vector (array) columns to vector (array) of subrecords
PxMakeVect              = mv		; Convert columns into vector (array) of columns
PxMerge                 = mg		; Combine master with one or more update inputs
PxModify                = mo		; Change column attributes and/or number of columns
PxNetezza               = **		; Netezza data access
PxOdbc                  = od		; ODBC data access
PxOracle                = or		; Oracle data access
PxPeek                  = pk      ; Print row column values to either the job log or to a separate output link
PxPromoteSubRec         = ps		; Promote columns of a subrecord to top-level
PxRemDup                = rd		; Eliminate adjacent rows with identical values for the key column(s)
PxRowGenerator          = rg		; Produce a set of mock data based on the specified meta data
PxSVTransformer         = **		; Enter data transformations (BASIC)
PxSample                = sa		; Randomly subset input and distribute among outputs
PxSequentialFile        = sf		; Sequential file access
PxSort                  = so		; Sort rows by ascending/descending column value
PxSplitSubRec           = ss		; Convert vector (array) of subrecords to top-level vectors
PxSplitVect             = sv		; Promote vector (array) column to top-level columns
PxSurrogateKeyGenerator = sk		; Create new surrogate key columns to add to incoming rows
PxSwitch                = sw		; Assign input rows to output links based on selector values
PxSybase                = **		; SYBASE data access
PxTail                  = tl		; Copy specified number of trailing input rows to output
PxTeradata              = **		; Teradata data access
PxWriteRangeMap         = wr		; Create map to use with "range" partitioning
STPPX                   = sp		; The Stored Procedure stage.
SYBASEOCPX              = **		; Sybase data access
TDMLoadPX               = **		; Teradata MultiLoad/TPump/FastExport Stage.
TerabulkPX              = **		; Teradata Bulk Loader.
TeradataPX              = **		; Teradata data access
UDBLoadPX               = **		; IBM DB2 UDB Bulk Loader
XMLInputPX              = xi		; XML Input Stage
XMLOutputPX             = xo		; XML Output Stage
XMLTransformerPX        = xt		; XML Transformer Stage
XPSLoadPX               = **		; Informix XPS Bulk Loader
ftpPX                   = ft		; FTP data access
rdbloadPX               = **		; Redbrick Bulk Loader

J.

Cranie · Post by **Cranie** » Wed Apr 16, 2008 1:09 am

jhmckeever thanks for the info thats what I wanted. Thanks.

Steve

Cranie · Post by **Cranie** » Wed Apr 16, 2008 4:06 am

Just been running my script and some stages are missing from the given list (in know it was not complete).

Does anyone have a similar complete listing of OLEType to stage name.

Any help appreciated.

Thanks

ray.wurlod · Post by **ray.wurlod** » Wed Apr 16, 2008 5:48 am

Do you mean that jmckeever's list is incomplete, or some other list that you are using? What did you discern to be missing?

Cranie · Post by **Cranie** » Wed Apr 16, 2008 8:51 am

Sorry yes, the list jmckeever provided is incomplete. For example:

Code: Select all

CCustomInput
CHashedInput
CTrxOutput
CHashedOutput
CCustomOutput
CJSJobActivity
CJobDefn
CCustomStage
CTrxInput
CContainerView
CJSActivityOutput
CJSActivityInput
CODBCInput
CSeqInput
CContainerDefn
CSeqOutput

There are others this is just from one project. The idea is to be able to get a list of all the stages used in our projects. I think I may be looking at the list and the OLETypes wrong. I am assuming now:

CContainerView = Container in the PARALLEL section
CHashedOutput = CHashedFileStage

Is there a complete list available / discoverable anywhere?

Thanks

Steve

ray.wurlod · Post by **ray.wurlod** » Wed Apr 16, 2008 5:04 pm

Fair enough - obviously jmckeever's site doesn't use server jobs.

CCustomInput and CCustomOutput are generic names for plug-in stage types, so that might be an issue for you.

You should find most, if not all, of the class types in the DSR_ESPROPS.H header file in DSINCLUDE directory in your project.

Cranie · Post by **Cranie** » Thu Apr 17, 2008 3:27 am

Thanks Ray, that looks like what I need.

Cheers

deanwalker · Post by **deanwalker** » Thu Apr 17, 2008 4:24 am

Anything that ends Input or Output is likely to be the appropriate end of a Link.