how to 'diff' two versions of the same job

[idocrm]

All,
Is there a way/tool that I can use to diff two versions of the same job so that I know what the changes are? Thank you.

[kcbland]

If you have MetaStage, go post your question there.

If you don't have MetaStage, you'll have to build something. There's a lot of history on this board regarding generating job design reports/exports/etc using either custom job design reverse-engineering, XML exports, or the Reporting Assistant. You'll of course have to be using some sort of source code control tool so that you have baselines. Then, write some parser to do the comparison.

And of source, your comparison is only as good as you make it. There's a lot of nooks and crannies to hide logic in.

When you write it, please document and sell it.

[Peytot]

If you do not have MetaStage ...
You also can use Version Control (Keep the historical information for each job) if you put the good commentary if not, you are obliged to open your two jobs for comparison.

[kduke]

idocrm

I have designed this but not implemented in DwNav. In Reporting Assitant doc_tool.mdb it will take several queries. The derivations are in DSProperties along with a bunch of other things. The queries look like this:

Code: Select all

select 
  * 
from 
  DSProperties a,
  DSProperties b
where
  a.ProjectName = b.ProjectName
  and 'QaJobSequence' = 'QaJobSequence2'
  and a.ContainerName = b.ContainerName
  and a.StageName = b.StageName
  and a.ObjOwnerName = b.ObjOwnerName
  and a.ObjectName = b.ObjectName
  and a.ObjectType = b.ObjectType
  and a.LinkInpOutProp = b.LinkInpOutProp
  and a.PropName = b.PropName
  and a.LineNumber = b.LineNumber
  and a.PropValue <> b.PropValue

This will only get you changes. You still need new items or deleted items.

Code: Select all

select 
   a.ProjectName,
   a.JobName,
   a.ContainerName,
   a.StageName,
   a.ObjOwnerName,
   a.ObjectName,
   a.ObjectType,
   a.LinkInpOutProp,
   a.PropName,
   a.LineNumber,
   a.PropOrder,
   a.PropValue
from 
   DSProperties a
where
   a.JobName = 'QaJobSequence'
   and not exists (
      select 
         *
      from 
         DSProperties b
      where
         b.JobName = 'QaJobSequence2'
         and a.ProjectName = b.ProjectName
         and a.ContainerName = b.ContainerName
         and a.StageName = b.StageName
         and a.ObjOwnerName = b.ObjOwnerName
         and a.ObjectName = b.ObjectName
         and a.ObjectType = b.ObjectType
         and a.LinkInpOutProp = b.LinkInpOutProp
         and a.PropName = b.PropName
         and a.LineNumber = b.LineNumber
    )
;

This will get you most of it. You still need deleted items, stage and link changes. It routines change then that is a different issue.

Kim.

[idocrm]

Thanks.

[scboyce]

This is exactly the question one should be able to answer very quickly when managing code in terms of change control. This question is another reason I wrote ParseDSX.pl which is posted elsewhere in this forum. Look for "DSX Cutter" as the main thread title.

Basically, ParseDSX will extract individual jobs and routines out of DSX export files produced from Manager. Since ParseDSX will "mask" all unimportant variable metadata, it is very good for detecting "true" changes in a job or routine.

Simply export the job(s) using manager, run the export DSX files through ParseDSX.pl and then do file compares on the resulting DSX files. The "real" differences will become obvious. Using a good file compare tool really helps. I recommend ExamDiffPro which can be found here: http://www.prestosoft.com.

[kduke]

scboyce

You are diffing the DSX files? DSX files are hard to read. That might help you see on a global basis to see if the job has changed but I would not want to look for detail changes in a DSX diff.

Kim.

[scboyce]

Well obviously, the DSX file does contain alot of "gobbly gook", but usually, the differences show up as recognizable snippets of code that make sense to the developer. Most of the time the only difference is some mapping code or job control code. Try it, you would be surprised how readable it is.

[kcbland]

This is a next to impossible feat. While Kim and Steve have good attempts, the biggest problem lies in building change comparison. What I hope people see is that the design metaphor Ascential has chosen does not allow for a user to maintain a written history of revisions an object (job, routine, etc) has undergone.

Until version 5, there was not an annotation stage. Previously, I had recommended that customers use the objects long description dialog box to record a history of changes. For example:

Date | Developer Name | Issue Number | Version Number | Description
---------------------|----------------------------------------|--------------------------|---------------------------|---------------------------------------------------------------------------------------------
2003-03-21 | Ken Bland | <<none>> | 2.4 | Release 2.4 standard improvements
2003-04-29 | Steve Boyce | <<none>> | 2.6 | Added MDM instance connection parameters
2003-05-28 | Ken Bland | <<none>> | 2.6 | Modified to select the actual coupon number instead of UNA
2003-07-03 | Steve Boyce | <<none>> | 2.7 | Added new fields for AutoShip.
2003-07-17 | Steve Boyce | <<none>> | 2.7.x | Format DNIS code
2003-09-12 |Steve Boyce | <<none>> | 2.8.4 | Default NULL Dates to BusinessDate (modified to handle 0 dates)
2003-09-29 | Steve Boyce | <<none>> | 2.9.x | Activated new fields for AutoShip.

One of the problems with this approach is the Arial font that is not fixed width, therefore formatting is uneven and spacing is irregular. This is probably the best compromise. It is part of the SDLC (that's software design life cycle) that a developer write a technical design document to go with any creation/revision to a design, plus have complete job design history within a version control tool. Peer reviews and code walk-thrus are part of the life cycle. This is software development folks.

[Teej]

The problem with the annotiation stage is that it pretty much get out of control when there are many changes to the job.

It would be neat if DataStage would have an option to require you to update the job description every time you save a job, or at least release it to Version Control or something like that.

-T.J.

[kcbland]

(Exaggerated voice)

Anything would help.