XML Stage: Regroup problem for multilevel child elements

Maximus_Jack · Post by **Maximus_Jack** » Sun Feb 02, 2014 3:31 am

Hi
I'm using the XML stage( not the XML output), to create an XML file, i'm having a single input link, where the data is coming from a netezza database, which is having elements in multiple levels, meaning, each child element is having a sub child ( note: i'm getting a row each repeating elements and i need to break this repeating elements to form the XML,
the XML is having several multiple repeating elements)

as the data is coming from a single input link,i use a regroup stage, during the regroup process, where the parent and child elements are segregated, the immediate child is having all the sub child elements in its structure itself,

how can i segregate the sub child from the previous child structure?

I tried having a separate input link for each sub child and using another regroup i tried to tie the sub child to the previous child,
but in the scope section, i'm seeing only the 'top' not the previous regroup link

any ideas???

thanks
MJ

stuartjvnorton · Post by **stuartjvnorton** » Sun Feb 02, 2014 4:46 pm

Could you post a sample so we can see what you want to do?

eostic · Post by **eostic** » Sun Feb 02, 2014 9:49 pm

It sounds like you have all the nested groups coming in on a single link?......such as company, department, team, employee, task.....(All as perfect sugroups in a straight path?)

If so, just keep going.......in my example above, I would need 5 regroup steps.....each one with a scope of the one before it.....and the first one having the highest scope....

Ernie

Maximus_Jack · Post by **Maximus_Jack** » Mon Feb 03, 2014 3:46 pm

thanks for responding..

yes followed your suggestion and created a regroup for each level and the mapping also seems to be working, but when i execute the job i'm facing an error as below and the job aborts
XML_5,0: Message bundle error Can't find resource for bundle com.ibm.e2.Bundle_E2_engine_msgs_en_US, key E2IllegalStateException.parentCursorNotWritable

I googled and searched this forum, seems like everyone has used hjoin and other stuffs but my job doesn't have that, can you please help me on this

My job info
--------------
Seq File -- > transormer --> copy stage --> XML pack

One more question
----------------------
i understand, keys can only be picked from the parent, but can you please explain if there are more than one element in the parent based on what you will pick the key in the regroup step and if more than one elements in the parent are changing at different intervals

eostic · Post by **eostic** » Mon Feb 03, 2014 4:49 pm

...hard to say on the error. sounds like maybe it generated an error that it couldn't find the text for.

...best suggestion there is to start breaking down the assembly. I thought you mentioned that everything was on one incoming link....don't introduce hjoin at this point.....just get your nested hierarchy working first.

....

I don't really understand your other question, but if it is about nested keys, then you have to become very familiar with the hierarchy that you are trying to compose. In some situations, perhaps you need to create your own keys to identify different groupings if the lists in the xsd don't correspond directly to the nested values on your input.

Ernie

Maximus_Jack · Post by **Maximus_Jack** » Mon Feb 03, 2014 11:07 pm

yes, the input is coming from single link, as the XML is having several levels, i'm using 15 regroup steps to replicate the hieararchy/levels, i dont know whether this error due to the number of regroup steps i'm using.

i coouldn't understand from your post, what my approach should be to fix
this issue or my next step should be, can you please elaborate me more on that

thanks
MJ

Maximus_Jack · Post by **Maximus_Jack** » Tue Feb 04, 2014 4:02 am

After changing the data types of few elements and changing the keys, the job got executed, now what the problem i'm facing is...

Just for testing purpose, i'm sending only two records to the XML pack and i have changed changed the data for one element ( say 'country') in the penultimate level on the second record only , so that i could see whether those records in that level are repeating, and i kept that element as the key which i changed.

even after that i'm getting only one record in the XML file, that too only
the second record, the first record is missing.
FYI -- both the records are exactly the same until the "country" element

any thoughts?

eostic · Post by **eostic** » Tue Feb 04, 2014 8:54 am

it's "missing"? hmm. Could be a lot of things.

I'd probably start debugging it by removing the other value and see if you get "nothing" in the result.

Also check your validation settings....perhaps the value that you are using for your testing isn't accepted by an enumeration check or other validity check inside the xsd. ...and even then, you have to be careful...the xml stage is exclusively controlled by the xsd.

Ernie

Maximus_Jack · Post by **Maximus_Jack** » Tue Feb 04, 2014 9:02 am

few more testing outcomes...

FYI - the difference between the first and second record is, i just changed the data of one element from the first record, and pasted it as a second record and that is the key in the regroup steps.
result:1
--------
when i just put the first record, the job is running fine and the XML is generated.

Result:2
---------
when i just put the second record, the job is running fine and the XML is generated.

Result:3
---------
when i put the first record twice, its working fine and i'm getting just one record in the XML as expected

Result:4
---------
when i put the second record twice, its working fine and i'm getting just one record in the XML as expected

Result:5
---------
when i put both the first and second record, this is where the job is failing with the below error
XML_5,0: Message bundle error Can't find resource for bundle com.ibm.e2.Bundle_E2_engine_msgs_en_US, key E2IllegalStateException.parentCursorNotWritable

XML_5,0: Fatal Error: CDIER0832E: Internal error: Parent cursor not writable or the accessor has already been written to. step=Projection-165, item={urn:ibm:e2:operator:Op-165:output-port:output}top

eostic · Post by **eostic** » Tue Feb 04, 2014 9:06 am

are they in sorted order?

...and what does this mean:

when i put the first record twice, its working fine and i'm getting just one record in the XML as expected

I'm confused.....Are you aggregating or something....why wouldn't you get two lowest level records in the xml in this case (assuming this key is at the penultimate lowest)?

Ernie

Maximus_Jack · Post by **Maximus_Jack** » Tue Feb 04, 2014 9:37 am

yes they are sorted

Ernie, correct me if i'm wrong, if two records are exactly same, won't the XML combine them by itself based on the keys and just produce one record?

Code: Select all

      <PrefBObj>
         <row1/>
         <row2>S</row2>
         <row3>100006</row3>
         <Extension>
           <XEntity>
             <XComm/>
             <Date>2005-07-21 19:38:56</Date>
             <Typehh/>
             <UpdateDt>2006-03-10 05:28:22</UpdateDt>
           </XEntity>
         </Extension>
         <Type>100006</Type>
       </PrefBObj>

this is the one level of the XML i'm fiddling with, my expectation is, for every unique value of "row3", the elements row1 and row 2 should repeat if they are different from previous row1 and row2 values(case 1) if not just another row with row3 value(case2)
like below

CASE 1

Code: Select all

      <PrefBObj>
         <row1/>
         <row2>S</row2>
         <row3>100006</row3>
         <row1/>
         <row2>S</row2>
         <row3>100004</row3>
         <Extension>
           <XEntity>
             <XComm/>
             <Date>2005-07-21 19:38:56</Date>
             <Typehh/>
             <UpdateDt>2006-03-10 05:28:22</UpdateDt>
           </XEntity>
         </Extension>
         <Type>100006</Type>
       </PrefBObj>

CASE 2

Code: Select all

      <PrefBObj>
         <row1/>
         <row2>S</row2>
         <row3>100006</row3>
         <row3>100004</row3>
         <Extension>
           <XEntity>
             <XComm/>
             <Date>2005-07-21 19:38:56</Date>
             <Typehh/>
             <UpdateDt>2006-03-10 05:28:22</UpdateDt>
           </XEntity>
         </Extension>
         <Type>100006</Type>
       </PrefBObj>

few more info;
--i don't have an xsd, so i generated the xsd, from the sample file given.
--

Code: Select all

eostic · Post by **eostic** » Tue Feb 04, 2014 10:24 am

Ernie, correct me if i'm wrong, if two records are exactly same, won't the XML combine them by itself based on the keys and just produce one record?

Well....the problem really is just that we're doing this via text, and what is a "record" anyway?

In terms of a hierarchy, record can be a difficult thing to define.

Normally, the way I think of it, is if I have two incoming "rows", then somewhere in the resulting xml document, I am going to expect to see two lowest level repeating nodes.....maybe in the same tree, or one in one tree and one in another......

Whether those repeating nodes are in a single document and a single "row" going outbound is dependent upon other factors.

As to your example, the concern (based on our thread) is that elements row1, row2 and row3 are not nested elements.....row2 does not "belong" to row1, nor does row3 belong to row2. Elements row1 and row2 stand alone and are terminated --- they aren't "containing" the others. Based on what I see here, they are entirely independent nodes.

It could be that they all only occur once....or maybe each one occurs on its own.

...and Extension contains XEntity, etc. ....but Extension and Type are also, based on this sample, independent nodes.

Ernie

Maximus_Jack · Post by **Maximus_Jack** » Tue Feb 04, 2014 10:40 am

ok, based on the information i posted, any idea on why the error occurs, when the job process both the records at once?

eostic · Post by **eostic** » Tue Feb 04, 2014 12:19 pm

...it's hard to say at this point, because if they are truly independent, then the lengthy set of nested regroups is not going to be valid for this structure (or at least not for this portion of it).

row1 will be a list on its own

row2 will be a list on its own

row3 will be a list on its own

...each belonging (perhaps?) to the common key/node above them (so you would re-group down to their parent).

...then probably hjoin them based on that parent.
...then map them to the independent lists.

This may be where the "generated" xsd is not correct for what you want. Based on what I've read thus far, you should have a separate blue "list" icon for each of these. If not, then row1 (or row2 or row3) might not be properly defined in the xsd as a repeating list. That can happen if the "sample" document you were given is not a true representation of what the

Ernie