How do people here do code reviews.

afreas · Post by **afreas** » Mon Jan 12, 2004 10:08 am

We are starting a rollout of the Datastage tool across multiple development groups. I am wondering how people are doing code reviews for datastage server and parallel jobs. Thanks much.
ART

datastage · Post by **datastage** » Mon Jan 12, 2004 10:18 am

The code reviews I've seen have been typically simple. For the most part the developer of the job sits with another developer that understand the requirements of the job to some extent. The job developer gives a brief overview of what the job does, and if designed well with good naming standards the reviewer should understand what it does and can give input as to how the job might better be designed. If there is a separate group of people responsible for supporting DataStage jobs after they move to production then it makes sense to have a support member present as well.

I find the intent of the job review more to make sure the developer is using best practices and sticking with the company guidelines, to bounce off development ideas, and to have other eyes see the new code (theres always problems on ETL teams when each member knows their own work only and nothing of what the others are doing) than to verify it is completely correct. Verification for correctness is best done with a member of a QA team or business analyst and where the actual output of the job is verified against the expected output of the job.

Teej · Post by **Teej** » Mon Jan 12, 2004 4:08 pm

In order to do a code review, you needs to establish a Development Best Practice for DataStage PX and Server. This documentation is a living form, ever-changing with time and improving understanding of the tools for your task and environment.

For example, in PX for this location, lookups are frown upon here until 6.0.2 is deployed, which it should be vastly improved back to better than the Sort stage. We shift around vast quantity of data and encountered a performance issue with lookups especially with large reference datasets.

It does help that the code looks 'neat.' I am trying to instill the philosophy that a bigger set of small jobs are usually better than a single huge job, not only for performance reasons, but also for neatness reasons. It is easier to add a newly discovered requirement to a small job, than it is to ensure that a huge job is correctly formatted.

Commenting are also big here -- we use annotiations to clarify things ("This performs a lookup using the following keys: CLIENT_FOOBAR, MASTER_BARBOY, and GENERIC_WHOOPS") Yes, you can open the lookup stage to find out this, but this is a great tool to visually say the requirements. Code reviews can easily catch disrepencies between comments and code if this is practiced.

We also want to minimize database accesses. Code Review can help us by noticing whether we are pulling from the same table more than once, and can be simplified for performance reasons. Performance of the jobs are very big here at this company.

So in a nutshell -- identify what is important for your company. Establish a basic Best Practice methodology that are not too overbearing, and ensure that the Code Review validate the Best Practice methods you chose to follow. Also ensure that you are flexible enough to change Best Practices in light of new discoveries.

-T.J.

ray.wurlod · Post by **ray.wurlod** » Mon Jan 12, 2004 9:20 pm

One of the services the consultants on the Forum provide is a "health check". That is, an assessment of how well you are following best practices, and probably suggestions about how things can be improved.