Run on Single node in multi node environment

Shruthi · Post by **Shruthi** » Wed Dec 10, 2008 9:45 am

Hi,

Is there any parameter that can be defined at job level to make the job run on single node in multi-node (4 nodes) environment?
We dont have access to change the Config file. Is there any other parameter available in datastaga?

I'm facing issue with join stage which gives different output for each run in a multi-node environment but works fine with single node. Not able to trace this issue. So, want to run it on any one node out of the 4 nodes.

kandyshandy · Post by **kandyshandy** » Wed Dec 10, 2008 9:57 am

I am not sure whether this will help you. You don't have to logon to UNIX server.

Check Tools menu in Designer and click "Configurations", then you can create new config file and save it. In the job where you want to use this single node file, just goto job properties and click 'add environment variable' and then select $APT_CONFIG_FILE to specify the new config file

Shruthi · Post by **Shruthi** » Wed Dec 10, 2008 10:03 am

Thanks so much. Do you know where this new config file which we create in designer will be stored? In case if it gets created in unix, I just wanted to make sure that no other user deletes it.

kandyshandy · Post by **kandyshandy** » Wed Dec 10, 2008 10:16 am

it will be created at the same place as the default.apt . So you may want to keep others informed of the same

kandyshandy · Post by **kandyshandy** » Wed Dec 10, 2008 10:18 am

it will be created at the same place as the default.apt . So you may want to keep others informed of the same

Shruthi · Post by **Shruthi** » Wed Dec 10, 2008 10:18 am

Thanks.

prasadduvasi · Post by **prasadduvasi** » Wed Dec 10, 2008 10:22 am

You can create different config files with different number of nodes (single node,2 nodes,4 nodes) at the fallowing location

/opt/datastage/Ascential/DataStage/Configurations on your UNIX box

and use these different config files for different jobs in same project using the $APT_CONFIG_FILE environment variable

ray.wurlod · Post by **ray.wurlod** » Wed Dec 10, 2008 1:48 pm

Another way is to compile the job in trace mode, which then gives the ability to force all stages (operators) to execute in sequential mode. When you're done, re-compile not in trace mode.

balajisr · Post by **balajisr** » Wed Dec 10, 2008 11:02 pm

Shruthi,

If this issue is same as your previous thread regarding Join stage then my advice of running in single node is for debug purposes only. I advise you to find the root cause of the problem.

It is very difficult for me sitting in a remote location to find an exact cause of your issue. I cannot reproduce the same here.

For the time being,why don't you try running this job using default or Auto partition method which usually will be much faster than running the same in sequential mode and verify the results. Once you identify the root cause you can modify the original job.

Shruthi · Post by **Shruthi** » Thu Dec 11, 2008 12:31 am

We tried auto partition also. It didnt work. As this issue occurs only with huge data, we are not able to trace back to the root cause. We are still debugging the issue. Meanwhile, due to time constraint, planning to proceed with single node.

nani0907 · Post by **nani0907** » Thu Dec 11, 2008 2:38 am

Run the Job in sequential mode or do hash partitioning on the key column beacause it makes sure that the same key values are processed on the same node. check it once.

nani0907 · Post by **nani0907** » Thu Dec 11, 2008 2:39 am

Run the Job in sequential mode or do hash partitioning on the key column beacause it makes sure that the same key values are processed on the same node. check it once.

abhishekachrekar · Post by **abhishekachrekar** » Thu Dec 11, 2008 4:18 am

Yes, Hash partitioning on both the input links will solve this issue.