Passing Job Params and User Vars to Java Transformer Stage
Moderators: chulett, rschirm, roy
Passing Job Params and User Vars to Java Transformer Stage
Hi,
I've got a simple sequence that has a UserVariables activity setting some variables. Then there is a JobActivity which invokes a parallel job which has a couple of Java Transformer activities.
I want to be able to pass a user variable to the Java Transformer and I want to pass Job variables to the Java class as well, e.g. $JobName for logging and logic purposes.
I have tried adding things like #MyTransformer.$JobName# to the the java transformer's user properties, but the data doesn't get expanded. I've also tried to set a JVM option as -D#MyTransformer.$JobName# but that doesn't work either.
Also, I have noticed that if I have more than 1 java transformer step and I set a system property on just one, every java transformer step sees the system property. I think that this might be a bug.
Can anyone help regarding the variables - I really need to be able to pass them into the java code.
Many thanks,
Dave
I've got a simple sequence that has a UserVariables activity setting some variables. Then there is a JobActivity which invokes a parallel job which has a couple of Java Transformer activities.
I want to be able to pass a user variable to the Java Transformer and I want to pass Job variables to the Java class as well, e.g. $JobName for logging and logic purposes.
I have tried adding things like #MyTransformer.$JobName# to the the java transformer's user properties, but the data doesn't get expanded. I've also tried to set a JVM option as -D#MyTransformer.$JobName# but that doesn't work either.
Also, I have noticed that if I have more than 1 java transformer step and I set a system property on just one, every java transformer step sees the system property. I think that this might be a bug.
Can anyone help regarding the variables - I really need to be able to pass them into the java code.
Many thanks,
Dave
Thanks Mike,
I've tried using the macros in the text area where you can specify free text, i.e. properties, XML, whatever - the stuff that is accessed by getUserProperties() - and I've also tried to pass them through as JVM args but they don't get expanded - they just appear as the macro names when they hit the java code. There's nothing in the API to get the names of the stage or the links, etc. so I'm struggling.
Anyone else any ideas?
Many thanks,
Dave
I've tried using the macros in the text area where you can specify free text, i.e. properties, XML, whatever - the stuff that is accessed by getUserProperties() - and I've also tried to pass them through as JVM args but they don't get expanded - they just appear as the macro names when they hit the java code. There's nothing in the API to get the names of the stage or the links, etc. so I'm struggling.
Anyone else any ideas?
Many thanks,
Dave
Here's a chunk of code with a way that I have done it.......inside the properties box of the JavaTransformer I'll have something like:
initialContextFactory=myContextFactory
queueName=myQueue
...and then the code in my class is as below. To get it to work I had to play around with byte streaming. It was a bit of trial and error several years ago when I figured this out, but have used it since without a problem.
private QueueConnection theConnection;
private QueueSession theSession;
private Queue theQueue;
private QueueReceiver theReceiver;
private QueueSender theSender;
private TopicConnection theTConnection;
private TopicSession theTSession;
private Topic theTopic;
private TopicSubscriber theSubscriber;
private TopicPublisher thePublisher;
private String queueOrTopic;
private TextMessage myMessage;
public void initialize()
{
trace("*------------------ Custom Class Specific Trace Info -------------------------*");
trace(" ");
trace("Beginning Initialization...");
//*------------------------------------------------------------------------------------*
// Obtain user properties from the Stage level property "box" of the plugin GUI
//
// What kind of Links are calling me? (am I a source or a target?)
//*------------------------------------------------------------------------------------*
char cr = '\r' ;
char lf = '\n';
byte[] userProperties=getUserProperties().getBytes();
InputStream propertyStream=new ByteArrayInputStream(userProperties);
Properties properties = new Properties();
try{
properties.load(propertyStream);
}
catch(IOException e){
e.printStackTrace();
}
//properties that must be supplied by the DataStage Developer
String initialContextFactory = properties.getProperty("initialContextFactory");
String providerURL = properties.getProperty("providerURL");
String connectionFactory = properties.getProperty("connectionFactory");
String queueName = properties.getProperty("queueName");
queueOrTopic = properties.getProperty("queueOrTopic");
if (initialContextFactory == null)
{
info("Value missing. Property syntax should be: initialContextFactory=<initialContextNamingFactory>");
}
if (providerURL == null)
{
info("Value missing. Property syntax should be: providerURL=<providerURL>");
}
if (connectionFactory == null)
{
info("Value missing. Property syntax should be: connectionFactory=<connectionFactory>");
}
if (queueName == null)
{
info("Value missing. Property syntax should be: queueName=<SourceOrTargetQueueName>");
initialContextFactory=myContextFactory
queueName=myQueue
...and then the code in my class is as below. To get it to work I had to play around with byte streaming. It was a bit of trial and error several years ago when I figured this out, but have used it since without a problem.
private QueueConnection theConnection;
private QueueSession theSession;
private Queue theQueue;
private QueueReceiver theReceiver;
private QueueSender theSender;
private TopicConnection theTConnection;
private TopicSession theTSession;
private Topic theTopic;
private TopicSubscriber theSubscriber;
private TopicPublisher thePublisher;
private String queueOrTopic;
private TextMessage myMessage;
public void initialize()
{
trace("*------------------ Custom Class Specific Trace Info -------------------------*");
trace(" ");
trace("Beginning Initialization...");
//*------------------------------------------------------------------------------------*
// Obtain user properties from the Stage level property "box" of the plugin GUI
//
// What kind of Links are calling me? (am I a source or a target?)
//*------------------------------------------------------------------------------------*
char cr = '\r' ;
char lf = '\n';
byte[] userProperties=getUserProperties().getBytes();
InputStream propertyStream=new ByteArrayInputStream(userProperties);
Properties properties = new Properties();
try{
properties.load(propertyStream);
}
catch(IOException e){
e.printStackTrace();
}
//properties that must be supplied by the DataStage Developer
String initialContextFactory = properties.getProperty("initialContextFactory");
String providerURL = properties.getProperty("providerURL");
String connectionFactory = properties.getProperty("connectionFactory");
String queueName = properties.getProperty("queueName");
queueOrTopic = properties.getProperty("queueOrTopic");
if (initialContextFactory == null)
{
info("Value missing. Property syntax should be: initialContextFactory=<initialContextNamingFactory>");
}
if (providerURL == null)
{
info("Value missing. Property syntax should be: providerURL=<providerURL>");
}
if (connectionFactory == null)
{
info("Value missing. Property syntax should be: connectionFactory=<connectionFactory>");
}
if (queueName == null)
{
info("Value missing. Property syntax should be: queueName=<SourceOrTargetQueueName>");
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
added note to code and example above...I should have been more explicit about Job Parameters, since that was the topic...... in the properties box, using the code above, values can be passed in as:
initialContextFactory=#myContextFactory#
queueName=#myQueue#
Ernie
initialContextFactory=#myContextFactory#
queueName=#myQueue#
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Hi all,
I've successfully passed the job params in via the java transformer properties and have used them to set the classpath and jvm options across all of the transformers as this is common. Thanks for your help with this.
Is there a way of getting hold of the name of the stage that is running? i.e. if I have a stage called MyTrans which is a Java Transformer, is there a key that I can enter that will allow MyTrans to be passed to the Java code in the properties?
Many thanks,
Dave
I've successfully passed the job params in via the java transformer properties and have used them to set the classpath and jvm options across all of the transformers as this is common. Thanks for your help with this.
Is there a way of getting hold of the name of the stage that is running? i.e. if I have a stage called MyTrans which is a Java Transformer, is there a key that I can enter that will allow MyTrans to be passed to the Java code in the properties?
Many thanks,
Dave
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
DSGetStageInfo(DSJ.ME, DSJ.STAGENAME) will return it, but I'm not sure how to invoke DataStage API functions from the Java Transformer stage.
There may be some read-only properties in the Properties collection; you will need to research this for yourself.
There may be some read-only properties in the Properties collection; you will need to research this for yourself.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
That's not a piece of run time metadata that is easily available.....can you describe what it is that you are trying to accomplish? The name of a Stage itself is not always that meaningful....
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Basically, I'm chaining together a number of Java transformer stages as the majority of the work that my client has done for processing the data in the past has been done in Java.
A number of the Java stages are generic and reusable and some are repeated across the job. Each step is named appropriately, for example Stage1, Stage2, Stage3, Stage4, etc.
What I want to do is to add this information to the logs / audit trail that the Java code creates such that we can see which stage the piece of code is executing in so that if there is a problem we can tie it down to a particular point in the job.
But I want to be able to do this without having to add and maintain a property on every stage saying stageName=Stage2, etc.
Does that make sense?
Many thanks.
A number of the Java stages are generic and reusable and some are repeated across the job. Each step is named appropriately, for example Stage1, Stage2, Stage3, Stage4, etc.
What I want to do is to add this information to the logs / audit trail that the Java code creates such that we can see which stage the piece of code is executing in so that if there is a problem we can tie it down to a particular point in the job.
But I want to be able to do this without having to add and maintain a property on every stage saying stageName=Stage2, etc.
Does that make sense?
Many thanks.
Sure.... I understand. Makes sense. Wish you could easily pull out the value. The JavaPack API was designed without exposure to all functions in the normal api, otherwise it would be there. There are a few other things I would have liked to see also, such as the ability to pull Data Elements for each column in addition to name and datatype. Ah, well.
An alternative, while not quite so elegant, would be to drop a transformer in between each, and have it grab "its" Stage name and pass it along in a column on the link. You could discern the order that way, provided the same naming style and convention is used.
Ernie
An alternative, while not quite so elegant, would be to drop a transformer in between each, and have it grab "its" Stage name and pass it along in a column on the link. You could discern the order that way, provided the same naming style and convention is used.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>