Concatening XML lines into one chunk

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Concatening XML lines into one chunk

Post by clarcombe »

For my sins I am working back at the company who tortured me with XML in January.

I have rows with a transaction id and a field which contains the XML
<PAIEMENT> other elements <\PAIEMENT>

The object of the exercise is to end up with an XML file in the format

Code: Select all

<Start> 
          <PAIEMENT> 
           other elements 
          <\PAIEMENT> 
          <PAIEMENT> 
           other elements 
           <\PAIEMENT> 
<\Start>


However what I get is

Code: Select all

<Start> 
          <PAIEMENT> 
           other elements 
          <\PAIEMENT> 
<\Start> 
<Start> 
          <PAIEMENT> 
           other elements 
          <\PAIEMENT> 
<\Start> 
What I thought I could do was output this using XML Output to a chunk and then add the <Start> flags but I have been unable to achieve this.

Any suggestions
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
mayura
Participant
Posts: 40
Joined: Fri Aug 01, 2008 5:58 am
Location: Mumbai

Re: Concatening XML lines into one chunk

Post by mayura »

In transformation before genarating xml in constraints write @OUTROWNUM=1 AND @PARTITIONNUM=0.
May be this wil helpful.try it.

Thanks,
Mayura :idea:
Mayura
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Both formats (the one you are getting and the one you want) without more detail at this point seem kinda screwy. Why is <PAIEMENT> in your first example repeated? Are the elements beneath it repeated? Uniquely identified? The elements beneath it ought to be the repeating element, and aggregation should naturally produce one /start and one /paiment and not repeat "either".

Looks like another scenario where the consuming tool doesn't really know xml, and as such, is doing something physical with the paiment elements (counting them, or parsing them manually, etc.).....

Unless........the exception to this might be if there is some detail missing...does PAIMENT have an "attribute" value? Does paiment in each case have dramatically different sub elements? [maybe paiment can be "credit card" and in another instance, be "check", etc.]....

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

PS. "For my sins I'm working back at the company who tortured me with XML".....

:)

Oh, come on, XML is fun! And now that Ray and IOD let the cat out of the bag, it's going to get even more fun [and better] in the near future.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

The XML originates in this format

Code: Select all

<Start>
	<PAIEMENT>
		<E1_ENTITE>ENTLEGACY</E1_ENTITE>
		<E1_CODE_REGLMNT>E  </E1_CODE_REGLMNT>
		<E1_MONTANT>250  </E1_MONTANT>
		<E1_MODE_PAIEMENT>VIR</E1_MODE_PAIEMENT>
		<E1_DEVISE_REGLEMENT>EUR</E1_DEVISE_REGLEMENT>
		<E1_TAUX_CONV>1.00  </E1_TAUX_CONV>
		<E1_DT_RECEPTION>31/10/2009  </E1_DT_RECEPTION>
		<E1_IBAN>MIDGL2050505  </E1_IBAN>
		<CLIENT>
			<E1_CODE_CLIENT>CLILEGACY  </E1_CODE_CLIENT>
		</CLIENT>
		<CLIENT>
			<E1_CODE_CLIENT>CLILEGACY   </E1_CODE_CLIENT>
		</CLIENT>
		<PIECE>
			<E1_CODE_ATTRIBUT_REF>FL    </E1_CODE_ATTRIBUT_REF>
			<E1_REFERENCE>FACLEGACY   </E1_REFERENCE>
			<E1_MONTANT>200    </E1_MONTANT>
		</PIECE>
		<PIECE>
			<E1_CODE_ATTRIBUT_REF>AL    </E1_CODE_ATTRIBUT_REF>
			<E1_REFERENCE>AFFLEGACY   </E1_REFERENCE>
			<E1_MONTANT>50    </E1_MONTANT>
		</PIECE>
	</PAIEMENT>
                <PAIEMENT> ..... 
What I do is remove the top level start and take the paiementS and then separate them into transactions at the Xpath level of Start/PAIEMENT , hence the transaction id.

All the transactions are then stored in a dataset and only the valid transactions (determined later) are used .The invalid ones are be used to reconstruct a valid XML file to be passed onto error processing. This is the part I am having difficulty with.

One of the problems is that I have created this XML from a very dodgy specification and it could actually be incorrect.However in its present form it is valid XML

The unique attribute is the E1_CODE_REGLEMENT and indeed I use this further along the job. But for the invalid transaction processing, I don't want to have to extract all the fields just to add the <Start> <\Start> flags

Is this any clearer ?

Are there any dates on when this becomes more fun ? :?
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

I'm kind of lost as to the ultimate goal and what things look like....are you creating this from a set of rows coming in, or reading it, or both?
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
clarcombe
Premium Member
Premium Member
Posts: 515
Joined: Wed Jun 08, 2005 9:54 am
Location: Europe

Post by clarcombe »

The ultimate goal is to have a final file which is the same as the original less the valid transactions

Original XML File

Code: Select all

<START>
     <PAIEMENT>
          transaction 1
     <\PAIEMENT>
     <PAIEMENT>
          transaction 2
     <\PAIEMENT>
     <PAIEMENT>
          transaction 3
     <\PAIEMENT>
<\START>
Intermediate Dataset

Code: Select all

Transaction Id 1, <PAIEMENT> transaction 1 <\PAIEMENT>
Transaction Id 2, <PAIEMENT> transaction 2 <\PAIEMENT>
Transaction Id 3, <PAIEMENT> transaction 2 <\PAIEMENT>
Various processing to extract valid transactions

Final Error XML File

Code: Select all

<START>
     <PAIEMENT>
          transaction 2
     <\PAIEMENT>
     <PAIEMENT>
          transaction 3
     <\PAIEMENT>
<\START>
Colin Larcombe
-------------------

Certified IBM Infosphere Datastage Developer
pillip
Premium Member
Premium Member
Posts: 50
Joined: Thu Dec 10, 2009 10:43 am

Concatenating XML lines into chunk.

Post by pillip »

I have a similar issue.
I have used the output XML column as '/' (root) to make an XML chunk and then used a transformer to concatenate this with the start and end tags. Used another XML output stage.
But still i am unable to get the required output.

This is what i m expecting
<Start tag> XML Chunk </Start tag>

Thanks.
clarcombe wrote:The ultimate goal is to have a final file which is the same as the original less the valid transactions

Original XML File

Code: Select all

<START>
     <PAIEMENT>
          transaction 1
     <\PAIEMENT>
     <PAIEMENT>
          transaction 2
     <\PAIEMENT>
     <PAIEMENT>
          transaction 3
     <\PAIEMENT>
<\START>
Intermediate Dataset

Code: Select all

Transaction Id 1, <PAIEMENT> transaction 1 <\PAIEMENT>
Transaction Id 2, <PAIEMENT> transaction 2 <\PAIEMENT>
Transaction Id 3, <PAIEMENT> transaction 2 <\PAIEMENT>
Various processing to extract valid transactions

Final Error XML File

Code: Select all

<START>
     <PAIEMENT>
          transaction 2
     <\PAIEMENT>
     <PAIEMENT>
          transaction 3
     <\PAIEMENT>
<\START>
Post Reply