passing XML data to a file.

rajan.n · Post by **rajan.n** » Fri Oct 13, 2006 4:08 am

Hi frnz,
Am using a before sub routine to create a xml file for the job details of a job which is given at run time, and in the job am retriving the data writen to xml file and passing them to respective files,
my job works fine for some jobs but not for all. the fatal error wich i get is."XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting", this is the first stage in my job and this is xml file stage. the delimiter is given as "NULL" in the format , this works for some jobs.and data is aslo passed to reqired files. can any body have exp on this so kindly let me know ..the reason y its giving this warning and any solutions for the same,
Thanks much in advance.
Rajan.

chulett · Post by **chulett** » Fri Oct 13, 2006 7:46 am

Which XML stage are you using? There's no 'XML file stage'...

rajan.n · Post by **rajan.n** » Fri Oct 13, 2006 8:15 am

ya , am sorry thats a sequential file am using.

ray.wurlod · Post by **ray.wurlod** » Fri Oct 13, 2006 3:48 pm

In that case you will need to incorporate adding tags into your own derivation logic.

rajan.n · Post by **rajan.n** » Mon Oct 16, 2006 7:21 am

Hi Ray,
can u pls describe it clearly..? and am not clear regarding using that tags how can we do that ,and if u have any doc's regarding this can u pls fwd to me.
thnx much in advance.

ray.wurlod wrote:In that case you will need to incorporate adding tags into your own derivation logic. ...

ray.wurlod · Post by **ray.wurlod** » Mon Oct 16, 2006 9:20 pm

It means that you need to generate output with tags.
For example, the derivation of LastName might be

Code: Select all

"<LastName>":InLink.LastName:"</LastName>"

rajan.n · Post by **rajan.n** » Mon Oct 16, 2006 10:28 pm

Hi ray ,
thanks a lot for the responce.
my job seems like this:

. . . . . . . . . . . . . . . . .|---->transfrmer----database
seq--->xmlinput -->|---->transformer---seq1
. . . . . . . .. . . . . . . . . .|---->transformer---seq2
. . . . . . . . . . . . . . . . . .|---->transformer---seq3

so , am reading my xml file from first seq file , there it self its giving the error,i can see the data is not passing to the xmlinput.
for some jobs it work fine but not for all. it gives the fatal error and aborts.
the error is :

XML,0: Error reading on import.
XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting
XML,0: Import error at record 0.
XML,0: The runLocally() of the operator failed.

i have checked all the options what all present in the seq file.

can u pls tell me where i need to give this tag.i have tried this giving in all the transformers,but didtn worked.

ray.wurlod wrote:It means that you need to generate output with tags.
For example, the derivation of LastName might be
Code: Select all
"<LastName>":InLink.LastName:"</LastName>"
...

rameshrr3 · Post by **rameshrr3** » Mon Oct 16, 2006 11:50 pm

Have you tried reading the input file from a folder stage, and passing its contents as the XML source column? We have quite a few jobs doing the same and it works fine.Has Xpath expression been defined in xml input stage column definition.

rajan.n · Post by **rajan.n** » Tue Oct 17, 2006 12:31 am

Hi ramesh,
thanks for ur reply,
i dint see any folder stage in my pallette can u pls tell me where can i get it from .am in px jobs.

rameshrr3 wrote:Have you tried reading the input file from a folder stage, and passing its contents as the XML source column? We have quite a few jobs doing the same and it works fine.Has Xpath expression been defined in xml input stage column definition.

Kirtikumar · Post by **Kirtikumar** » Tue Oct 17, 2006 2:39 am

Very first thing, folder stage is not available in PX. So you have to use some alternative for that.

Are you sure about the sequential file is reading the whole XML data as a single record and in a single column?
As mentioned in your first post, if the XML is generated in before job, then tags can be added as follows:

Code: Select all

Seq File --> Transformar--->XML.....(rest of job)

In tranformer add the derivattion mentioned in Ray's post. Once you have done this, do not forget to change the XPaths in XML stage to have this root tag.

If this thing is not working, pls post the XML generated and some of the columns with their XPaths, then it will be helpful for others to understand.the problem

rajan.n · Post by **rajan.n** » Tue Oct 17, 2006 3:12 am

Hi Kirti kumar,
thanks for the responce.
my seq file is reading the whole XML as a single record and in a single column"CONTENT"
as u said i placed a transformer in between the seq and XMLinput.
and used the ray's code like "<CONTENT>":CONTENT:"</CONTENT>" and inthe XML input stage , i had mannually typed this colomn name CONTENT,as u said to "do not forget to change the XPaths in XML stage " remaining are existed as usual, and even this did not worked. regarding the xml created by the before subroutine ,
<?xml version="1.0" encoding="UTF-8"?>
<Job Name="ctry_cmpny_usr_role_lkp"
Desc=""
Status="2"
WaveNo="1"
StartDateTime="2006-08-21T12:05:01"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:04"
ElapsedSecs="4">
<ParamSet>
<Param Name="$BAMSSFEEAM"
Type="0"
Desc=""
Value="CFMBMS"/>
<Param Name="$CFMUSERID"
Type="0"
Desc=""
Value="cfmrun"/>
<Param Name="$CFMPASSWD"
Type="1"
Desc=""
Value="*****"/>
<Param Name="$CFMDB"
Type="0"
Desc=""
Value="cfmdb"/>
<Param Name="$DSPATH"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/DSEngine/bin"/>
<Param Name="JOBNAME"
Type="0"
Desc=""
Value="ctry_cmpny_usr_role_lkp"/>
<Param Name="$CFMSCHEMA"
Type="0"
Desc=""
Value="CFMSTG"/>
<Param Name="$DSLOGDIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/logs"/>
<Param Name="$DSDATADIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/data"/>
<Param Name="$DSPROJECT"
Type="0"
Desc=""
Value="CFM"/>
<Param Name="$DSSERVER"
Type="0"
Desc=""
Value="proctected"/>
<Param Name="$DSUSERID"
Type="0"
Desc=""
Value="abc"/>
<Param Name="$DSPASSWD"
Type="1"
Desc=""
Value="********"/>
</ParamSet>
<ComponentSet>
<Stage Name="INPUT_SELECT"
StageStatus="10"
StageType="DSDB2PX"
Desc=""
StartDateTime="2006-08-21T12:05:05"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:00"
ElapsedSecs="0">
<OutputLinks>
<Link Name="Lkp1"
LinkType="3"
Desc=""
Stage="T1"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="T1"
StageStatus="10"
StageType="CTransformerStage"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp1"
LinkType="1"
Desc=""
Stage="INPUT_SELECT"/>
</InputLinks>
<OutputLinks>
<Link Name="Lkp2"
LinkType="3"
Desc=""
Stage="OUTPUT_ADD"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="OUTPUT_ADD"
StageStatus="10"
StageType="PxLookupFileSet"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp2"
LinkType="1"
Desc=""
Stage="T1"/>
</InputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
</ComponentSet>
</Job>

THIS IS THE ONE FOR ONE OF THE JOB , CREATED BY THE BEFORE SUB ROUTINE....AND NOT WORKING.BUT SOME ARE WORKING FINE.
once again thanks a lot and waiting for ur reply.

Kirtikumar wrote:Very first thing, folder stage is not available in PX. So you have to use some alternative for that.

Are you sure about the sequential file is reading the whole XML data as a single record and in a single column?
As mentioned in your first post, if the XML is generated in before job, then tags can be added as follows:
Code: Select all
Seq File --> Transformar--->XML.....(rest of job)
In tranformer add the derivattion mentioned in Ray's post. Once you have done this, do not forget to change the XPaths in XML stage to have this root tag.

If this thing is not working, pls post the XML generated and some of the columns with their XPaths, then it will be helpful for others to understand.the problem

Kirtikumar · Post by **Kirtikumar** » Tue Oct 17, 2006 6:40 am

Hey you have not posted the XPaths used in your job.

Nways, instead of parsing all the columns from the XML try n parse single or less number of cols wherever you have problem.
So say first only try n extract JobName - the XPath for it should be /Job/@Name.

chulett · Post by **chulett** » Tue Oct 17, 2006 6:49 am

I thought there was some way in PX to have the Sequential File stage return only the fully qualified pathname of the file found. Rather than trying to make it do the folder trick of 'reading the entire file into one record' - which while it may work sometimes will never work 100% of the time - see if that can be done.

Then switch the XML Input stage to source using the URL/Filepath option.