setting User Status in a parallel job

[asorrell]

I've searched for quite a bit, and I've seen a lot of references to setting User Status in a parallel job (for retrieval from a job sequence), but nothing about the mechanics of doing this.

How's it done?

I have a job where a stored procedure stage returns a run number that needs to be passed to other jobs later in the sequence, and I think the User Status would be the easiest way, though I'm open to suggestions.

By the way - I can't just dump it to a file and then read it from the sequence because the solution has to be generic enough to add to a lot of sequences without modification.

[chulett]

My understanding is you'd need to use a BASIC Transformer to do that. Either that or it's not possible.

[asorrell]

Ugh. Not really what I wanted. I think you also might be able to do it via some sort of interim parallel routine that calls the DSSetUserStatus, but I was also hoping to avoid writing any code for this.

[chulett]

I'll be curious what we find out, I'm just going by what I've read here and have no way to do any actual testing of this.

[ray.wurlod]

The user status value is stored in one of the RT_... hashed files in the local DataStage repositories. I have not been able to find anything in the DataStage C API for accessing it. Would a BASIC Transformer be really that bad?

[asorrell]

*Gasps in HoRRoR!*

Did I just hear Ray "Stay away from the Basic Transformer" Wurlod ask me if "using a Basic Transformer would be so bad"?



I guess not....

I'm also looking at the "triggers" section of a transformer stage. It seems to allow setting some sort of user variables, though I've not done it before.

I'll post what I've tried later on.

Thanks!

[ray.wurlod]

My doctor told me to stay away from cigarettes, so I bought a cigarette holder.

In general avoid BASIC Transformer because of their overhead, but if there's no other way... (Like avoiding sorting unless it's necessary.)

The Variables you found in the stage properties are actually the stage variables, so that won't help. Triggers are additional pieces of C++ code that are fired because of processing in the Transformer stage; unless you can find a C-based technique for updating the user status area then that won't help either.

(You could, of course, create some C code that uses UVread and UVwrite via system calls to update the record containing the job's user status area, but that won't give you much in the way of performance.)

[evee1]

Hi Andy,
What did you end up doing?

I have a similar issue. I'm implementing a generic sequence to cater for various steps of file validation. One of them is a routine that verifies the total of a given source field in a file against the total given in a footer.
I need to pass the result of a validation back up to the parent sequence and I was planning to set the UserStatus in a BASIC transformer and then query the status in a sequence.

I can't test it though as there is no DSSetUserStatus routine available in my temporary DS installation (I'm waiting for a proper environment to be installed).

Also what did you do with the output from the BASIC transformer? Did you need it for anything. I don't, as I only care about comparing the stats and returning the result. So in my job I have a BASIC transformer stage that calls (or rather will call, once it is actually available ) DSSetUserStatus routine, but then I need some output from the transformer. Otherwise the job won't compile. I created a dummy outpt Seq file, but I'm not really fond of it.

[DSguru2B]

I once attempted, poorly, to accomplish this. Did not use it and have not gotten a chance to enhance it either. You are more than welcomed to see if it can provide a starting point. The code lies here.
I would advise to get rid of the malloc() command as it provided a lot of grief in pxEreplace().

[DSguru2B]

Another way to do it would be to call the stored proc and assign the value to an environment variable. This way you can leverage the run number in all your jobs.

[BI-RMA]

I tried to use a Routine to set the UserStatus variable of a PX-Job from within a BASIC-Transformer. It just did not work.

Not only do you get a warning in the job-log:
(BASIC_Transformer_14).#0.BASIC_Transformer_14 (DSSetUserStatus): Error reading Job Status record. JobNo = 2336 JobName = Test_CDC.#0 WaveNo = 1.

The UserStatus variable remains empty for the job, so follow-up jobs do not get their parameter-values. Unfortunately it looks like You can only make use of the UserStatus variable from within Server jobs.

[magma]

Well, I did use a couple of suggestions here to build a work-around. Seems you *can* call a Server Routine and/or access a Server Hash from within a Parallel Job after all.

Absolutely a work-around only (as it relies on current Hash structures), but it gets us past this particular problem.

[magma]

A couple of people have asked what we did... as I said prev. it is in kludge to fit an existing need. If we were designing from scratch, we'd go another way. Also it was an exercise in "can we do it?"

<Before>

SourceStage ----> <Transformer> ----> SequentialFile

written as a Server Job. The transformer would call a user-written routine which invoked DSSetUserStatus. The last row down the Source link would become the actual Job's $UserStatus.
The Sequential file was an append to /dev/null (or NUL if DOS).


<After>

SourceStage ----> ServerSharedContainer

written as a Parallel Job, using a Sever Shared Container with 1 input field. Set the Advanced Properties to 'Sequential' (to avoid Parallel nodes). The Shared Container has the following.

InputLink -----> Transformer -------> SequentialFile

Again, the Sequential File is /dev/null. The Basic Transformer calls user-written code which *pokes* the supplied value directly into the Job's control file (as a Hash File). We couldn't use DSSetUserStatus due to a known problem with Parallel Jobs.

Might look messy, but this was quite useful. The Server Shared Container gave us the ability to call BASIC routines (or use Hash Files). Because it was Shared, we were able to plug it in to a number of other Jobs.

What do you think?

[ray.wurlod]

Once you know the location of the user status area - I'm sure I've documented this somewhere on DSXchange - you can write a parallel routine that uses InterCall functions ic_open, ic_recordlock, ic_writev and ic_close to update that particular field (field #9 in one of the RT_STATUSnnn records).

You can find the InterCall Developer's Guide in any set of UniVerse manuals, whether from IBM or Rocket Software, for example here. InterCall has not changed much in the last 20 years.

[asorrell]

Thanks to magma for giving more details as to how he got his solution to work. Changing topic status to "WorkAround"!