usage of Common block across transformer

[talk2shaanc]

Hey Guys,

In the below routine, if i pass a value it stores it in common and if i give argument values as 'Get', it returns the old value stored in COMMON.

The below routine works for a process i.e., in a transformer when i keep switching between arguments values.

But i wanted to store the value in one transformer and retrieve the value in another transformer.

Read in the manual that i need to use something like
$INCLUDE InitializeToCommon.H in my routine.
But, if i use the above line directly in my code, it would give error: Unable to open DSU_BP/InitializeToCommon.H

Code: Select all

InitializeToCommon(Arg1)
-----------------------------------------------
      Common /InitializeToCommon/ Initialize, LastValue
      RoutineName='InitializeToCommon'

Ans = ''
If Arg1='Get' Then
Ans=LastValue
End Else
      If Initialize Then
         Ans=''
         LastValue=Arg1
      End Else
         Initialize = 1
         
         LastValue = Arg1
      End
End

Can anyone write me a steps how to do it with some modification to above routine?

[roy]

Hi,
Under Routines\sdk\RowProc you can find similar routines that come with the product.
(at least for 7.5)

IHTH,

[ArndW]

The COMMON block works wonderfully well when you are working within one process; but common is not shared across processes. So your method would only work if both transformers are actually in the same process. In Server this is the case, so it will work for (I just wrote a test job to make sure that this is still the case).

The $INCLUDE statement inserts a file's contents into a job source file during compilation; in your case you do not necessarily need to use an INCLUDE file.

You can write a function that includes your code and call that function.

But for simplicity it might make more sense to use a stage variable and pass that downstream to the other transformers - this is how the previous record value is often exchanged between transform stages.

[talk2shaanc]

My actual requirement is :

Intialize it in one of the transformer and use it in after-job routine of the same job.

So just for testing the function , i created a dummy job.

seqfile --->Trans1--->Hashfile--->trans2---->seq
in trans1 i am initializing and trans2 i am trying to fetch that value. But finally in my target sequential file I am getting the null values.

The manual says, if you want to use in two stages , i need to use $INCLUDE.
But I dont know how. can anyone give me the steps or Pseudo code for the same?


I have further refined my routine as below;

Code: Select all

Common /InitializeToCommon/ Initialize, LastValue
      RoutineName='InitializeToCommon'

      Ans = ''
      If Initialize Then
         If Arg1='Get' Then
            Ans=LastValue
         End Else
            Ans=''
            LastValue=Arg1
         End
      End Else
         Initialize = 1

         LastValue = Arg1
      End

[talk2shaanc]

Hi,
Under Routines\sdk\RowProc you can find similar routines that come with the product.
(at least for 7.5)

IHTH,

Thanks, for your suggestion. But I have already done that.
Nowdays, I always do R&D, and search first, and then if i dont get anything and my deadline is close, I knock the doors of DSXchange.

[ArndW]

You do not need to use $INCLUDE.

just make sure that in both places you define the COMMON block with the same name and the same number of variables (yes, you can change the names around, common blocks are interpreted positionally).

The manual might state that it makes sense to use $INCLUDE; the syntax is $INCLUDE <filename> <recordname>; usually the <filename> is a directory and the <recordname> is a file in that directory. You will need to have an entry in your VOC for this file; you can use "dsinclude" and put your include there, but I remember having problems in the past when doing a project upgrade and losing the include file.

If you can, do a "CREATE.FILE INCLUDE 19" and that will create a sub-directory called INCLUDE into which you can copy your InitializeToCommon.H file and declare it as

Code: Select all

$INCLUDE INCLUDE InitializeToCommon.H

[chulett]

My actual requirement is: Intialize it in one of the transformer and use it in after-job routine of the same job.

You can't. That's quite different from initializing in one transformer and using in anther.

[ArndW]

Craig,

bingo, I missed that.

[Sainath.Srinivasan]

If you want it in post job, you need to set the status or land it in any file.

[talk2shaanc]

You can't. That's quite different from initializing in one transformer and using in anther.

Hey, craig i didnt get you. You mean to say it cannot be accomplished?
Well I think, I can do it by using $INCLUDE in my current routine, for my actual requirement.

After reading everybody post and some search on dsxchange for the same, I am bit surprised , why my current routine is returning NULL, for my dummy job.

Code: Select all

seq--->tr1-->hsh-->tr2--->seq

[crouse]

There is a caveat in the release notes since about release 5 or 6 that cautions about the use of COMMON storage BETWEEN stages. Since the introduction of row buffering, results are unpredictable if your processing depends on COMMON storage populated in one transform and retreived in another, if row buffering is enabled.

Row buffering, by default, is turned off at the project level. It can be turned on/off at the project and at the job level.

Row buffering, if enabled, USUALLY improves performance, so lots of folks automatically enable it. While it can improve performance, there are cases where it gives unpredictable results in fairly complex jobs.

But, if you know where the stones are, it can appear you are walking on water. Good luck and watch your step.

-Craig

[ArndW]

talk2shaanc,

I'm glad you are confident that you can do it. I am not, I just wrote a test job and the COMMON is not shared between the job and it's after-job (or before-job) routine; these are different processes.

So you will have to explicitly pass the values in through the after-job call or write the data to a file, the COMMON method will not work.

[talk2shaanc]

There is a caveat in the release notes since about release 5 or 6 that cautions about the use of COMMON storage BETWEEN stages. Since the introduction of row buffering, results are unpredictable if your processing depends on COMMON storage populated in one transform and retreived in another, if row buffering is enabled.

Row buffering, by default, is turned off at the project level. It can be turned on/off at the project and at the job level.

Row buffering, if enabled, USUALLY improves performance, so lots of folks automatically enable it. While it can improve performance, there are cases where it gives unpredictable results in fairly complex jobs.

But, if you know where the stones are, it can appear you are walking on water. Good luck and watch your step.

-Craig

I am aware of this fact too, I read it in one of the Old thread. But in my dummy job or for the project, Row buffering Enabling/Disabling didnt make any difference.

[crouse]

Never say never. (I realize you didn't, but the cliche fits )

Usually we test with small amounts of data. But, under production volumes the results could prove differently. The caveat is there for a reason.

Best of luck.

-Craig

[ArndW]

Shantanu,

see a bit earlier for the use of the $INCLUDE {file} {include} statement. I was surprised that you could skip the {file} part, but in the BASIC manual it states

When program is specified without filename, program must be a record in the same
file as the program currently containing the $INCLUDE statement.

I recommend to use a specific directory.

The help text shows the use of $INCLUDE as a sample of how to do it; but again you can just use identical code wherever you wish to use the COMMON (I agree that using $INCLUDE is the structured approach to doing this).

The example chooses the Subroutine(Arg1,ErrorCode) which is the format of a before/after subroutine call and is extremely misleading. If they called the routines BeforeSub or AfterSub then I would complain, but as it stands there's no mention that they are such.

Even at the first versions of DS these jobs were seperate processes and COMmon wouldn't transfer to them.