Output dataset not partitioned as expected
Moderators: chulett, rschirm, roy
Output dataset not partitioned as expected
I have set up a APT_CONFIG_FILE with 2 nodes, and a job writing to a dataset from a Transformer stage (Execution mode = Parallel). The dataset shows only 1 partition (expected 2 partitions).
I made another job with a Change Capture stage, writing to a dataset. It also shows only 1 partition.
Did I miss a prerequisite ? Is there another environment variable to set ?
Thanks in advance
I made another job with a Change Capture stage, writing to a dataset. It also shows only 1 partition.
Did I miss a prerequisite ? Is there another environment variable to set ?
Thanks in advance
Thuong
best regards
best regards
Look into your director log for an entry starting "Environment variable settings..." and in the details for that entry see which APT_CONFIG file was actually used for the run - is it the 2-node you expected? I suspect not.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
In that case, how are you determining that there is only 1 partition in your dataset? I would use the Director -> Tools -> Dataset Management to examine the dataset.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Thanks for your answer.ArndW wrote:In that case, how are you determining that there is only 1 partition in your dataset? I would use the Director -> Tools -> Dataset Management to examine the dataset.
I have also used Designer-> Tools -> Dataset Management and checked that all records are written on 1 node only.
Another colleague told me to reboot the server. I have to test this, but I am currently at a remote location.
Thuong
best regards
best regards
Perhaps you specified the output dataset to run "sequentially", this would explain your issue as well. I doubt that rebooting the server would affect this particular problem, but it can't hurt it, either.
Roy's suggestion of "Having you tried turning it off and on again?" certainly works in many cases. Moss's recommendation of "Have you tried forcing an unexpected reboot?" usually doesn't
Roy's suggestion of "Having you tried turning it off and on again?" certainly works in many cases. Moss's recommendation of "Have you tried forcing an unexpected reboot?" usually doesn't
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
The execution mode is set to Parallel. Thanks for your input.ArndW wrote:Perhaps you specified the output dataset to run "sequentially", this would explain your issue as well. I doubt that rebooting the server would affect this particular problem, but it can't hurt it, either.
Roy's suggestion of "Having you tried turning it off and on again?" certainly works in many cases. Moss's recommendation of "Have you tried forcing an unexpected reboot?" usually doesn't
Thuong
best regards
best regards
Open up a job monitor from the director after the job has run and set it to show instances. Does it show 2 instances for the transform stage?
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Thanks, Mike.Mike wrote:Post your 2 node configuration file as well. I saw this behavior once with what appeared to be a 2 node configuration at first glance. Ended up that there was an extra set of curly braces misplaced that essentially caused the second node to be ignored.
Mike
You are probably right. Here is my initial syntax:
{
node "node1"
{
fastname "BERNINI3"
pools ""
resource disk "E:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "E:/IBM/InformationServer/Server/Scratch" {pools ""}
}
}
{
node "node2"
{
fastname "BERNINI3"
pools ""
resource disk "E:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "E:/IBM/InformationServer/Server/Scratch" {pools ""}
}
}
It should be:
{
node "node1"
{
fastname "BERNINI3"
pools ""
resource disk "E:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "E:/IBM/InformationServer/Server/Scratch" {pools ""}
}
node "node2"
{
fastname "BERNINI3"
pools ""
resource disk "E:/IBM/InformationServer/Server/Datasets" {pools ""}
resource scratchdisk "E:/IBM/InformationServer/Server/Scratch" {pools ""}
}
}
Thuong
best regards
best regards