use user defined configuration file

abyss · Post by **abyss** » Tue Oct 13, 2015 9:52 pm

hi all
another basic question
I create a configuration file in project dir e:/project_a/config/projectA_conf.apt

and I changed the project environment variables APT_CONFIG_FILE to e:/project_a/config. then the jobs just don't run, it can compile

when i check the tools->configerations...... on designer, it still read the default config file that located under installation directory, and it can't pass the check.

what else i need to change, anyone can help?

thanks
abyss

abyss · Post by **abyss** » Wed Oct 14, 2015 12:03 am

after install the datastage, I just create the project in the default directory, but now I create new project in the new file structure and this how the problem start. must be something wrong.

after i export the project environment variable to file, i can see the following:

Code: Select all

[EnvVarDefns]
...
APT_CONFIG_FILE\Parallel\3\FilePath\D:/IBM/InformationServer/Server/Configurations/default.apt\3\Project\Configuration file\The Parallel job configuration file.
...
APT_ORCHHOME\Parallel\3\DirPath\D:/IBM/InformationServer/Server/PXEngine\3\Project\Parallel home directory\Parallel home directory path.
...

[EnvVarValues]
"APT_CONFIG_FILE"\1\"E:/projectA_master/projectA_plus/configurations/default.apt"
...

so i assume the setup is correct?

ray.wurlod · Post by **ray.wurlod** » Wed Oct 14, 2015 12:03 am

APT_CONFIG_FILE needs the pathname of the configuration file, not just a parent directory. Conventionally the file name has the ".apt" suffix.

It seems to me that the $APT_CONFIG_FILE environment variable value is not a job parameter in your job design.

abyss · Post by **abyss** » Wed Oct 14, 2015 5:30 am

thanks ray, my membership expired, i will get a new one tomorrow

abyss · Post by **abyss** » Sun Oct 18, 2015 11:20 pm

hi ray
how to add $APT_CONFIG_FILE environment variable value into the job?

i add the env variables via job properties -> parameters -> add environment variable...
then i add the $APT_CONFIG_FILE on the list, what should i do nect.

thanks
abyss

abyss · Post by **abyss** » Mon Oct 19, 2015 12:03 am

my job is really simple, just a row gen stage and link to peek stage.
in the tool -> configuration panel doesn't show the config file except the default one in the installation directory (I want the job use the config file defined in project directory)

and it can not pass the validation

Code: Select all

##I IIS-DSEE-TFSC-00001 16:56:43(004) <main_program> APT configuration file: D:/IBM/InformationServer/Server/Configurations/default.apt
##E IIS-DSEE-TFIO-00211 16:56:49(000) <APT_RealFileExportOperator in APT_FileExportOperator,0> APT_Communicator::connectTo: connect() failed due to Unix error = 10,061 (Unknown error) on node IS912ENGDEV using ConnectionInfo object 'TCP,  connection Host: IS912ENGDEV.ad.uow.edu.au (130.130.210.80), TCP port number: 0', RETRYING connect() 
##E IIS-DSEE-TFIO-00211 16:56:50(000) <APT_RealFileExportOperator in APT_FileExportOperator,0> APT_Communicator::connectTo: connect() failed due to Unix error = 10,061 (Unknown error) on node IS912ENGDEV using ConnectionInfo object 'TCP,  connection Host: IS912ENGDEV.ad.uow.edu.au (130.130.210.80), TCP port number: 0', RETRYING connect() 
##F IIS-DSEE-TFIO-00112 16:56:50(001) <APT_RealFileExportOperator in APT_FileExportOperator,0> Fatal Error: APT_Communicator::pmSendPartitionInfo() failed on node IS912ENGDEV for partition 0 of dataset 0 (write failed to handle 4) Bad file descriptor
##E IIS-DSEE-TFPM-00192 16:56:50(000) <node_node1> Player 2 terminated unexpectedly.
##E IIS-DSEE-TFPM-00338 16:56:50(000) <main_program> APT_PMsectionLeader(1, node1), player 2 - Unexpected exit status 1.
##E IIS-DSEE-TFSC-00011 16:56:55(000) <main_program> Step execution finished with status = FAILED.
##E IIS-DSEE-TCOA-00069 16:56:55(001) <main_program> ERROR: check configuration file failed.

ray.wurlod · Post by **ray.wurlod** » Mon Oct 19, 2015 1:35 am

To add an environment variable as a job parameter go to Job Properties, Parameters tab, and click the Add Environment Variable button. Choose $APT_CONFIG_FILE from those on offer.

When the job run request is made, provide the full pathname of your configuration file as the value for the $APT_CONFIG_FILE parameter.

abyss · Post by **abyss** » Mon Oct 19, 2015 6:07 am

thanks so much

I did all that and set the full path of the config file as default value, but still give me the error.

this is really simple stuff but still can't get it right. uhh

chulett · Post by **chulett** » Mon Oct 19, 2015 7:26 am

Then it would seem your configuration file itself could be the source of the problem, perhaps you should post it? Not sure how much people could help not knowing your topology, but worth a shot.

abyss · Post by **abyss** » Mon Oct 19, 2015 6:36 pm

this is my config file (windows os), really simple:

Code: Select all

{
	node "node1"
	{
		fastname "IS912ENGDEV"
		pools ""
		resource disk "E:/project_A_master/project_A_plus/datasets/datasegments" {pools ""}
		resource scratchdisk "E:/scratch" {pools ""}
	}
	node "node2"
	{
		fastname "IS912ENGDEV"
		pools ""
		resource disk "E:/project_A_master/project_A_plus/datasets/datasegments" {pools ""}
		resource scratchdisk "E:/scratch" {pools ""}
	}
}

we use dedicated engine tier topology, so engine tier on one vm and service/metadata tier on the other vm.

I create another project just use default setting (create a project under IS installation dir, use the default scratch disk under installation directory etc, etc) the job runs fine.

another thing i can't understand is: i cannot see the config file on the designer client except the default one, and even the default one cannot pass the validation.

cheers

abyss · Post by **abyss** » Mon Oct 19, 2015 6:47 pm

the other thing i noticed is the job can compile but can NOT run
after job failed, you must recompile the job again because is 'not in a runable stage'

should i leave the config file in default dir? (<is_install_dir>/InformationServer/Server/Configurations)

ray.wurlod · Post by **ray.wurlod** » Mon Oct 19, 2015 10:03 pm

Just as a test can you place your configuration file in the same Configurations folder as default.apt, then use the Configurations tool in DataStage Designer to verify it, letting us know the results?

abyss · Post by **abyss** » Mon Oct 19, 2015 10:38 pm

Hi ray:
I create 2 new config files in the default default.apt directory. one is the config file i created for project A and another .apt file is exact same copy of default.apt

I login to project_A, i can see both config files, and either of them can NOT pass the validation and then I login to another project that created in the default directory under IS installation dir, both of them CAN pass the validation

priyadarshikunal · Post by **priyadarshikunal** » Tue Oct 20, 2015 12:57 am

did you verify the permissions on configuration files?

abyss · Post by **abyss** » Tue Oct 20, 2015 6:56 pm

Hi
I fixed the problem, the key is set $APT_USE_IPV4=1 as the default in projects in the administrator console.

Hosts should have at least the following 2 entries:

# localhost name resolution is handled within DNS itself.
127.0.0.1 localhost
::1 localhost

You should be able to ping localhost via V4 and V6 IP topologies:
ping -4 localhost
ping -6 localhost

Set $APT_USE_IPV4=1 as the default in all projects or set this as Windows default environment.
Steps to set $APT_USE_IPV4 DataStage Administrator client:
Select Projects tab
Select Project
Click Properties
Click Environment button
Click Parallel Category
Locate APT_USE_IPV4 and set to TRUE

this only need for the project not defined in the default directory (which are most of them).

regards
abyss