Standardization:Using the seq file extractd from xml
Standardization:Using the seq file extractd from xml
While running the above job, I am getting the following errors:
Data,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 1 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected exit status 1. [processmgr/slprocess.C:368]
SelectMX,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 3 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 3 - Unexpected exit status 1. [processmgr/slprocess.C:368]
APT_PMsectionLeader(1, node1), player 4 - Unexpected exit status 1. [processmgr/slprocess.C:368]
main_program: Step execution finished with status = FAILED. [sc/sc_api.C:242]
However, I do not get any error( the job runs successfully) when I take the input data as a sequential file with data extracted from the SQL database.
What should I modify to run the above mentioned job perfectly.
Data,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 1 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected exit status 1. [processmgr/slprocess.C:368]
SelectMX,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 3 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 3 - Unexpected exit status 1. [processmgr/slprocess.C:368]
APT_PMsectionLeader(1, node1), player 4 - Unexpected exit status 1. [processmgr/slprocess.C:368]
main_program: Step execution finished with status = FAILED. [sc/sc_api.C:242]
However, I do not get any error( the job runs successfully) when I take the input data as a sequential file with data extracted from the SQL database.
What should I modify to run the above mentioned job perfectly.
tell us lots more. What are the sources/targets, what stage types, what are you trying to do, etc..... there's not enough context here to know what might be the solution.
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
I converted an xml to a sequential file using the job which looks like :eostic wrote:tell us lots more. What are the sources/targets, what stage types, what are you trying to do, etc..... there's not enough context here to know what might be the solution.
Ernie
Sequential File(Input as xml) XML Input Stage Sequential file(Output File B)
Later on I tried to standardize the home address data in the Output File B, where the job looks like:
Sequential File(Input as File B) Standardize(Country) Filter(Select Country) Standardize(Home Address using MXADDR.set) Sequential File
While running the above job, I am getting the following errors:
Data,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 1 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected exit status 1. [processmgr/slprocess.C:368]
SelectMX,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 3 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 3 - Unexpected exit status 1. [processmgr/slprocess.C:368]
APT_PMsectionLeader(1, node1), player 4 - Unexpected exit status 1. [processmgr/slprocess.C:368]
main_program: Step execution finished with status = FAILED. [sc/sc_api.C:242]
However, I do not get any error( the job runs successfully) when I take the input data as a sequential file with data extracted from the SQL database.
What should I modify to run the above mentioned job perfectly.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
How much have you tested the first job? Is the file you created exactly as you expected? Are the formats everything that you need? Sounds like the file is not getting created as you expected, or that the metadata in the second Job is not correct when reading that file. Have you used the Sequential stage before in a job like your second one? Have you looked at the file in an editor, and tried it with just a few rows? Can you create the file for the second job by hand and get that to work?
The problem (if it is a problem) may be in the first Job, but it's not clear if it's a problem, or just a mismatch between what the first job is doing and what you are trying to do with that data in the second.
Ernie
The problem (if it is a problem) may be in the first Job, but it's not clear if it's a problem, or just a mismatch between what the first job is doing and what you are trying to do with that data in the second.
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Yes I am getting the file in the expected format ( I suppose! For the xpath /TCRMService/RequestControl/DWLControl/requesterName , Output column is like <requesterName>MDMUSER</requesterName>eostic wrote:How much have you tested the first job? Is the file you created exactly as you expected? Are the formats everything that you need? Sounds like the file is not getting created as you expected, or that the metadata in the second Job is not correct when reading that file. Have you used the Sequential stage before in a job like your second one? Have you looked at the file in an editor, and tried it with just a few rows? Can you create the file for the second job by hand and get that to work?
The problem (if it is a problem) may be in the first Job, but it's not clear if it's a problem, or just a mismatch between what the first job is doing and what you are trying to do with that data in the second.
Ernie
And for xpath
/TCRMService/RequestControl/DWLControl/requesterName/text(), output column is like MDMUSER)
Yes, I have used the Sequential stage before in a job like my second one. However there is no difference in the metadata used in the first and second job, because ,
a). I have tried bifurcating the job into Job 1 and 2 as explained
b). I made the job as one and ran it
For both the cases the Error is same:
Case 1:
main_program: IBM WebSphere DataStage Enterprise Edition 8.1.0.5040
main_program: conductor uname: -s=AIX; -r=3; -v=5; -n=bcbeta052; -m=00C6BDFF4C00
main_program: orchgeneral: loaded
main_program: Echo:
main_program: Explanation:
main_program: Dump:
main_program: APT configuration file:
main_program: This step has 3 datasets:
main_program: Schemas:
Data_Set_80,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 1 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected exit status 1.
Copy_of_StandardizeCNTRY,0: Failure during execution of operator logic. [api/operator_rep.C:399]
Copy_of_StandardizeCNTRY,0: Input 0 consumed 0 records.
Copy_of_StandardizeCNTRY,0: Output 0 produced 0 records.
Case 2:
main_program: IBM WebSphere DataStage Enterprise Edition 8.1.0.5040
main_program: conductor uname: -s=AIX; -r=3; -v=5; -n=bcbeta052; -m=00C6BDFF4C00
main_program: orchgeneral: loaded
main_program: Echo:
main_program: Explanation:
main_program: Dump:
main_program: APT configuration file:
main_program: This step has 3 datasets:
main_program: Schemas:
StandardizeCNTRY,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
node_node1: Player 1 terminated unexpectedly. [processmgr/player.C:149]
main_program: APT_PMsectionLeader(1, node1), player 1 - Unexpected exit status 1.
SelectMX,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
xStan,0: Internal Error: (size <= blocksize_): iomgr/iocomm.C: 1736
main_program: Step execution finished with status = FAILED. [sc/sc_api.C:242]
In both the above case, there are few warnings like
When checking operator: When binding input interface field "ValorPiso_MXADDR" to field "ValorPiso_MXADDR": Implicit conversion from source type "ustring[max=10]" to result type "string[max=10]": Converting ustring to string using codepage ISO-8859-1. [api/interface_rep.C:6183]
For few fields generated out of standardize stage, but I have used the same job for
a). data extracted from a database to a sequential file.
It worked well. Only when I am extracting the data
b). From an xml file to a sequential file/data set,
the above errors are coming.
What can be the issue?
I also tried to change the environment variables for the
APT_DEFAULT_TRANSPORT_BLOCK:SIZE from 131072 to 262144 and
APT_DEFAULT_TRANSPORT_BLOCK:SIZE from 1048576 to 2097136
But because of the above changes few other parallel jobs stopped working. Hence I switched to default values.
Hard to say what to advise you here, except that you need to start breaking things down to find out where it works and where it blows up. Reduce it dramatically. Follow your XML stage with a transformer and exclude all but maybe 3 columns and maybe 1 row. Be certain that they are 100% character. See if that works...then add columns and rows back in...and so forth until you see what triggers the problem.
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Hi Ernieeostic wrote:Hard to say what to advise you here, except that you need to start breaking things down to find out where it works and where it blows up. Reduce it dramatically. Follow your XML stage with a transformer and exclude all but maybe 3 columns and maybe 1 row. Be certain that they are 100% character. See if that works...then add columns and rows back in...and so forth until you see what triggers the problem.
Ernie
Its solved now. In the env variables i changed
APT_DEFAULT_TRANSPORT_BLOCK_SIZE = APT_MAX_TRANSPORT_BLOCK_SIZE = 1048576
and it worked!
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
What were their previous values, who advised these changes, and (did they explain) why changing these values fixed the particular problem? For that matter, what was the real, underlying, original problem?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
The APT_DEFAULT_TRANSPORT_BLOCK_SIZE was 131072 initially which was not in line with the requisite value according to the formula for the block size. You can find the formula in the documentation. As of now i am not able to access the support site to retrieve that.ray.wurlod wrote:What were their previous values, who advised these changes, and (did they explain) why changing these values fixed the particular problem? For that matter, what was the real, underlying, original problem?