Source Code

[rcil]

Hi All,

Is there a way to copy all the user source code to a file? For example, there is a datastage job, with database as source using user defined sql and flat file as output. I want to extract all SQL from the source stage and any transformation logic in the transformer.

Is this possible? Is it possible to get the code from the dsx file? Your comments are greatly appreciated.

Thanks,
rcil

[kduke]

You can download JobReport.dsx from ADN. If you want it for all jobs and all routines then you need to download KgdGenHtml.zip. These generate html pages one per job and one per routine. JobReport.dsx generates the html for jobs. I wrote the one for routines. I wrote the code to run these for more than one job.

You could cut and paste it from the html or modify some of the code to get what you want. ASCL does not post the code for JobReport so that may not help. Most of my code is visable so you could modify it.

[ray.wurlod]

Server jobs are run by a DataStage BASIC program called DSD.RUN, for which source code is not available. It uses data (metadata?) in the RT_CONFIG file to figure out what to do, what dependencies there are, and so on.

Passive stage types expose method functions (some in BASIC, some in C) that are invoked by active stages. For example, the Sequential File stage has method functions for Open, Read, Write, Close, and some others for handling Abort conditions, etc. Source code for these is also unavailable. If you write your own plug-in stages, you would, naturally, have source code for those.

Server job Transfomer stage are invoked by a DataStage BASIC subroutine called DSD.StageRun. Again source code is not available, but the generated subroutine for each Transformer stage is visible in the RT_BPnnn directory, where nnn is the job number.

There is nothing in the product that allow you to document the end-to-end flow of ETL logic. A lineage analysis (produced in MetaStage) can do this kind of report, but is very limited in ways to customise the format of its reports.

[kduke]

Ray

I know you are correct but he said user source code. So I assumed he meant routines and batch jobs. He also said he wanted the SQL. Some this could be retrieved in Reporting Assistant. Hopefully my assumptions are correct.

[chulett]

Kim, I think you got it right.

The JobReport routine from ADN will give the OP what they are looking for based on how I read their post.

[rcil]

Kim,

Thanks for all the help. I was able to get all the source code what I want in html format. You added an excellent feature of getting the report for all jobs or by category. I noticed that you changed the JobReport input parameters.

The orginal JobReport takes three parameter and third one has multiple options but in the JobReport which I got from your zip file does not allow those options. Is there a way to add the options like to supress report sections 'SUPPSTGPROPS=1'

I tried couple of ways to do it like, I overwrite the JobReport with original JobReport which takes only three parameters and I changed the JobControl for 'KgdGenHtmlJobDocs'

Code: Select all

Deffun JobReport(JobName,OutputDirectory,SUPPSTGPROPS=1) calling "DSU.JobReport"

and Junk = JobReport(JobName, OutputDirectory,SUPPSTGPROPS=1)

but it did not work. The job was sucessfull but the option did not work (that option was to supress stage properties).

I tried in different way like

Code: Select all

Deffun JobReport(JobName,OutputDirectory,SUPPSTGPROPS) calling "DSU.JobReport"
* -----------------------------------------------------------------
* used in JobReport routine
* -----------------------------------------------------------------
      OutputDirectory = FilePath:ThisDay
      SUPPSTGPROPS = 1
and Junk = JobReport(JobName, OutputDirectory,SUPPSTGPROPS)

the job finished but the option did not supress the stage properties. Please can you help me in some how using those extra options available in JobReport routine.

Is there a way to create the directory anywhere you want using KgdMakeDir routine other than the project home directory? I mean if I want the output directory to be created on my home directory /home/rcil how can I do it.

thanks,
rcil