Passing Job Params and User Vars to Java Transformer Stage

[DHallam]

Hi,

I've got a simple sequence that has a UserVariables activity setting some variables. Then there is a JobActivity which invokes a parallel job which has a couple of Java Transformer activities.

I want to be able to pass a user variable to the Java Transformer and I want to pass Job variables to the Java class as well, e.g. $JobName for logging and logic purposes.

I have tried adding things like #MyTransformer.$JobName# to the the java transformer's user properties, but the data doesn't get expanded. I've also tried to set a JVM option as -D#MyTransformer.$JobName# but that doesn't work either.

Also, I have noticed that if I have more than 1 java transformer step and I set a system property on just one, every java transformer step sees the system property. I think that this might be a bug.

Can anyone help regarding the variables - I really need to be able to pass them into the java code.

Many thanks,

Dave

[Mike]

I'm not familiar with the java transformer, but the transformer stage allows you to use DataStage macros such as DSJobName.

Mike

[DHallam]

Thanks Mike,

I've tried using the macros in the text area where you can specify free text, i.e. properties, XML, whatever - the stuff that is accessed by getUserProperties() - and I've also tried to pass them through as JVM args but they don't get expanded - they just appear as the macro names when they hit the java code. There's nothing in the API to get the names of the stage or the links, etc. so I'm struggling.

Anyone else any ideas?

Many thanks,

Dave

[eostic]

Here's a chunk of code with a way that I have done it.......inside the properties box of the JavaTransformer I'll have something like:

initialContextFactory=myContextFactory
queueName=myQueue

...and then the code in my class is as below. To get it to work I had to play around with byte streaming. It was a bit of trial and error several years ago when I figured this out, but have used it since without a problem.

private QueueConnection theConnection;
private QueueSession theSession;
private Queue theQueue;
private QueueReceiver theReceiver;
private QueueSender theSender;

private TopicConnection theTConnection;
private TopicSession theTSession;
private Topic theTopic;
private TopicSubscriber theSubscriber;
private TopicPublisher thePublisher;

private String queueOrTopic;
private TextMessage myMessage;

public void initialize()
{

trace("*------------------ Custom Class Specific Trace Info -------------------------*");
trace(" ");
trace("Beginning Initialization...");
//*------------------------------------------------------------------------------------*
// Obtain user properties from the Stage level property "box" of the plugin GUI
//
// What kind of Links are calling me? (am I a source or a target?)
//*------------------------------------------------------------------------------------*
char cr = '\r' ;
char lf = '\n';

byte[] userProperties=getUserProperties().getBytes();
InputStream propertyStream=new ByteArrayInputStream(userProperties);
Properties properties = new Properties();


try{
properties.load(propertyStream);
}
catch(IOException e){
e.printStackTrace();
}

//properties that must be supplied by the DataStage Developer

String initialContextFactory = properties.getProperty("initialContextFactory");
String providerURL = properties.getProperty("providerURL");
String connectionFactory = properties.getProperty("connectionFactory");
String queueName = properties.getProperty("queueName");
queueOrTopic = properties.getProperty("queueOrTopic");

if (initialContextFactory == null)
{
info("Value missing. Property syntax should be: initialContextFactory=<initialContextNamingFactory>");
}
if (providerURL == null)
{
info("Value missing. Property syntax should be: providerURL=<providerURL>");
}
if (connectionFactory == null)
{
info("Value missing. Property syntax should be: connectionFactory=<connectionFactory>");
}
if (queueName == null)
{
info("Value missing. Property syntax should be: queueName=<SourceOrTargetQueueName>");

[lstsaur]

Dave,
What does exception error say? Just saying it doesn't work is hard to help.
Please also post your job design.

[eostic]

added note to code and example above...I should have been more explicit about Job Parameters, since that was the topic...... in the properties box, using the code above, values can be passed in as:

initialContextFactory=#myContextFactory#
queueName=#myQueue#

Ernie

[DHallam]

Many thanks for your responses - I'll try some stuff on Monday when I'm back in work and let you know how I get on and provide more detail, e.g. job design, etc.

[DHallam]

Hi all,

I've successfully passed the job params in via the java transformer properties and have used them to set the classpath and jvm options across all of the transformers as this is common. Thanks for your help with this.

Is there a way of getting hold of the name of the stage that is running? i.e. if I have a stage called MyTrans which is a Java Transformer, is there a key that I can enter that will allow MyTrans to be passed to the Java code in the properties?

Many thanks,

Dave

[ray.wurlod]

DSGetStageInfo(DSJ.ME, DSJ.STAGENAME) will return it, but I'm not sure how to invoke DataStage API functions from the Java Transformer stage.
There may be some read-only properties in the Properties collection; you will need to research this for yourself.

[DHallam]

There's nothing in the Java API to pull them out. I assume that I can just wrap the function up in # in the properties like

stageName=#DSGetStageInfo(DSJ.ME, DSJ.STAGENAME)#

Is that valid? I will try when back in work tomorrow.

Many thanks.

[ray.wurlod]

Alas, that is not valid. A parameter reference can not be a function call.

[eostic]

That's not a piece of run time metadata that is easily available.....can you describe what it is that you are trying to accomplish? The name of a Stage itself is not always that meaningful....

Ernie

[DHallam]

Basically, I'm chaining together a number of Java transformer stages as the majority of the work that my client has done for processing the data in the past has been done in Java.

A number of the Java stages are generic and reusable and some are repeated across the job. Each step is named appropriately, for example Stage1, Stage2, Stage3, Stage4, etc.

What I want to do is to add this information to the logs / audit trail that the Java code creates such that we can see which stage the piece of code is executing in so that if there is a problem we can tie it down to a particular point in the job.

But I want to be able to do this without having to add and maintain a property on every stage saying stageName=Stage2, etc.

Does that make sense?

Many thanks.

[eostic]

Sure.... I understand. Makes sense. Wish you could easily pull out the value. The JavaPack API was designed without exposure to all functions in the normal api, otherwise it would be there. There are a few other things I would have liked to see also, such as the ability to pull Data Elements for each column in addition to name and datatype. Ah, well.

An alternative, while not quite so elegant, would be to drop a transformer in between each, and have it grab "its" Stage name and pass it along in a column on the link. You could discern the order that way, provided the same naming style and convention is used.

Ernie

[DHallam]

Hi Ernie,

Many thanks for your help. I agree that the API is quite sparse. I'm maintaining a context across each stage as XML in a column on the links so I'll use that to track an index of how far along the job it is instead of the name.

Cheers,

Dave