passing XML data to a file.
Moderators: chulett, rschirm, roy
passing XML data to a file.
Hi frnz,
Am using a before sub routine to create a xml file for the job details of a job which is given at run time, and in the job am retriving the data writen to xml file and passing them to respective files,
my job works fine for some jobs but not for all. the fatal error wich i get is."XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting", this is the first stage in my job and this is xml file stage. the delimiter is given as "NULL" in the format , this works for some jobs.and data is aslo passed to reqired files. can any body have exp on this so kindly let me know ..the reason y its giving this warning and any solutions for the same,
Thanks much in advance.
Rajan.
Am using a before sub routine to create a xml file for the job details of a job which is given at run time, and in the job am retriving the data writen to xml file and passing them to respective files,
my job works fine for some jobs but not for all. the fatal error wich i get is."XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting", this is the first stage in my job and this is xml file stage. the delimiter is given as "NULL" in the format , this works for some jobs.and data is aslo passed to reqired files. can any body have exp on this so kindly let me know ..the reason y its giving this warning and any solutions for the same,
Thanks much in advance.
Rajan.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It means that you need to generate output with tags.
For example, the derivation of LastName might be
For example, the derivation of LastName might be
Code: Select all
"<LastName>":InLink.LastName:"</LastName>"
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi ray ,
thanks a lot for the responce.
my job seems like this:
. . . . . . . . . . . . . . . . .|---->transfrmer----database
seq--->xmlinput -->|---->transformer---seq1
. . . . . . . .. . . . . . . . . .|---->transformer---seq2
. . . . . . . . . . . . . . . . . .|---->transformer---seq3
so , am reading my xml file from first seq file , there it self its giving the error,i can see the data is not passing to the xmlinput.
for some jobs it work fine but not for all. it gives the fatal error and aborts.
the error is :
XML,0: Error reading on import.
XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting
XML,0: Import error at record 0.
XML,0: The runLocally() of the operator failed.
i have checked all the options what all present in the seq file.
can u pls tell me where i need to give this tag.i have tried this giving in all the transformers,but didtn worked.
thanks a lot for the responce.
my job seems like this:
. . . . . . . . . . . . . . . . .|---->transfrmer----database
seq--->xmlinput -->|---->transformer---seq1
. . . . . . . .. . . . . . . . . .|---->transformer---seq2
. . . . . . . . . . . . . . . . . .|---->transformer---seq3
so , am reading my xml file from first seq file , there it self its giving the error,i can see the data is not passing to the xmlinput.
for some jobs it work fine but not for all. it gives the fatal error and aborts.
the error is :
XML,0: Error reading on import.
XML,0: Consumed more than 100,000 bytes looking for record delimiter; aborting
XML,0: Import error at record 0.
XML,0: The runLocally() of the operator failed.
i have checked all the options what all present in the seq file.
can u pls tell me where i need to give this tag.i have tried this giving in all the transformers,but didtn worked.
ray.wurlod wrote:It means that you need to generate output with tags.
For example, the derivation of LastName might be...Code: Select all
"<LastName>":InLink.LastName:"</LastName>"
Last edited by rajan.n on Tue Oct 17, 2006 2:40 am, edited 1 time in total.
Hi ramesh,
thanks for ur reply,
i dint see any folder stage in my pallette can u pls tell me where can i get it from .am in px jobs.
thanks for ur reply,
i dint see any folder stage in my pallette can u pls tell me where can i get it from .am in px jobs.
rameshrr3 wrote:Have you tried reading the input file from a folder stage, and passing its contents as the XML source column? We have quite a few jobs doing the same and it works fine.Has Xpath expression been defined in xml input stage column definition.
-
- Participant
- Posts: 437
- Joined: Fri Oct 15, 2004 6:13 am
- Location: Pune, India
Very first thing, folder stage is not available in PX. So you have to use some alternative for that.
Are you sure about the sequential file is reading the whole XML data as a single record and in a single column?
As mentioned in your first post, if the XML is generated in before job, then tags can be added as follows:
In tranformer add the derivattion mentioned in Ray's post. Once you have done this, do not forget to change the XPaths in XML stage to have this root tag.
If this thing is not working, pls post the XML generated and some of the columns with their XPaths, then it will be helpful for others to understand.the problem
Are you sure about the sequential file is reading the whole XML data as a single record and in a single column?
As mentioned in your first post, if the XML is generated in before job, then tags can be added as follows:
Code: Select all
Seq File --> Transformar--->XML.....(rest of job)
If this thing is not working, pls post the XML generated and some of the columns with their XPaths, then it will be helpful for others to understand.the problem
Regards,
S. Kirtikumar.
S. Kirtikumar.
Hi Kirti kumar,
thanks for the responce.
my seq file is reading the whole XML as a single record and in a single column"CONTENT"
as u said i placed a transformer in between the seq and XMLinput.
and used the ray's code like "<CONTENT>":CONTENT:"</CONTENT>" and inthe XML input stage , i had mannually typed this colomn name CONTENT,as u said to "do not forget to change the XPaths in XML stage " remaining are existed as usual, and even this did not worked. regarding the xml created by the before subroutine ,
<?xml version="1.0" encoding="UTF-8"?>
<Job Name="ctry_cmpny_usr_role_lkp"
Desc=""
Status="2"
WaveNo="1"
StartDateTime="2006-08-21T12:05:01"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:04"
ElapsedSecs="4">
<ParamSet>
<Param Name="$BAMSSFEEAM"
Type="0"
Desc=""
Value="CFMBMS"/>
<Param Name="$CFMUSERID"
Type="0"
Desc=""
Value="cfmrun"/>
<Param Name="$CFMPASSWD"
Type="1"
Desc=""
Value="*****"/>
<Param Name="$CFMDB"
Type="0"
Desc=""
Value="cfmdb"/>
<Param Name="$DSPATH"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/DSEngine/bin"/>
<Param Name="JOBNAME"
Type="0"
Desc=""
Value="ctry_cmpny_usr_role_lkp"/>
<Param Name="$CFMSCHEMA"
Type="0"
Desc=""
Value="CFMSTG"/>
<Param Name="$DSLOGDIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/logs"/>
<Param Name="$DSDATADIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/data"/>
<Param Name="$DSPROJECT"
Type="0"
Desc=""
Value="CFM"/>
<Param Name="$DSSERVER"
Type="0"
Desc=""
Value="proctected"/>
<Param Name="$DSUSERID"
Type="0"
Desc=""
Value="abc"/>
<Param Name="$DSPASSWD"
Type="1"
Desc=""
Value="********"/>
</ParamSet>
<ComponentSet>
<Stage Name="INPUT_SELECT"
StageStatus="10"
StageType="DSDB2PX"
Desc=""
StartDateTime="2006-08-21T12:05:05"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:00"
ElapsedSecs="0">
<OutputLinks>
<Link Name="Lkp1"
LinkType="3"
Desc=""
Stage="T1"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="T1"
StageStatus="10"
StageType="CTransformerStage"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp1"
LinkType="1"
Desc=""
Stage="INPUT_SELECT"/>
</InputLinks>
<OutputLinks>
<Link Name="Lkp2"
LinkType="3"
Desc=""
Stage="OUTPUT_ADD"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="OUTPUT_ADD"
StageStatus="10"
StageType="PxLookupFileSet"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp2"
LinkType="1"
Desc=""
Stage="T1"/>
</InputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
</ComponentSet>
</Job>
THIS IS THE ONE FOR ONE OF THE JOB , CREATED BY THE BEFORE SUB ROUTINE....AND NOT WORKING.BUT SOME ARE WORKING FINE.
once again thanks a lot and waiting for ur reply.
thanks for the responce.
my seq file is reading the whole XML as a single record and in a single column"CONTENT"
as u said i placed a transformer in between the seq and XMLinput.
and used the ray's code like "<CONTENT>":CONTENT:"</CONTENT>" and inthe XML input stage , i had mannually typed this colomn name CONTENT,as u said to "do not forget to change the XPaths in XML stage " remaining are existed as usual, and even this did not worked. regarding the xml created by the before subroutine ,
<?xml version="1.0" encoding="UTF-8"?>
<Job Name="ctry_cmpny_usr_role_lkp"
Desc=""
Status="2"
WaveNo="1"
StartDateTime="2006-08-21T12:05:01"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:04"
ElapsedSecs="4">
<ParamSet>
<Param Name="$BAMSSFEEAM"
Type="0"
Desc=""
Value="CFMBMS"/>
<Param Name="$CFMUSERID"
Type="0"
Desc=""
Value="cfmrun"/>
<Param Name="$CFMPASSWD"
Type="1"
Desc=""
Value="*****"/>
<Param Name="$CFMDB"
Type="0"
Desc=""
Value="cfmdb"/>
<Param Name="$DSPATH"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/DSEngine/bin"/>
<Param Name="JOBNAME"
Type="0"
Desc=""
Value="ctry_cmpny_usr_role_lkp"/>
<Param Name="$CFMSCHEMA"
Type="0"
Desc=""
Value="CFMSTG"/>
<Param Name="$DSLOGDIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/logs"/>
<Param Name="$DSDATADIR"
Type="0"
Desc=""
Value="/home/dsadm/Ascential/DataStage/Projects/CFM/data"/>
<Param Name="$DSPROJECT"
Type="0"
Desc=""
Value="CFM"/>
<Param Name="$DSSERVER"
Type="0"
Desc=""
Value="proctected"/>
<Param Name="$DSUSERID"
Type="0"
Desc=""
Value="abc"/>
<Param Name="$DSPASSWD"
Type="1"
Desc=""
Value="********"/>
</ParamSet>
<ComponentSet>
<Stage Name="INPUT_SELECT"
StageStatus="10"
StageType="DSDB2PX"
Desc=""
StartDateTime="2006-08-21T12:05:05"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:00"
ElapsedSecs="0">
<OutputLinks>
<Link Name="Lkp1"
LinkType="3"
Desc=""
Stage="T1"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="T1"
StageStatus="10"
StageType="CTransformerStage"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp1"
LinkType="1"
Desc=""
Stage="INPUT_SELECT"/>
</InputLinks>
<OutputLinks>
<Link Name="Lkp2"
LinkType="3"
Desc=""
Stage="OUTPUT_ADD"/>
</OutputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp1"
RowCount="0"/>
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
<Stage Name="OUTPUT_ADD"
StageStatus="10"
StageType="PxLookupFileSet"
Desc=""
StartDateTime="2006-08-21T12:05:03"
EndDateTime="2006-08-21T12:05:05"
ElapsedTime="00:00:02"
ElapsedSecs="2">
<InputLinks>
<Link Name="Lkp2"
LinkType="1"
Desc=""
Stage="T1"/>
</InputLinks>
<InstanceSet>
<Instance Id="0"
CPU="0"
PID="0">
<Link Name="Lkp2"
RowCount="0"/>
</Instance>
</InstanceSet>
</Stage>
</ComponentSet>
</Job>
THIS IS THE ONE FOR ONE OF THE JOB , CREATED BY THE BEFORE SUB ROUTINE....AND NOT WORKING.BUT SOME ARE WORKING FINE.
once again thanks a lot and waiting for ur reply.
Kirtikumar wrote:Very first thing, folder stage is not available in PX. So you have to use some alternative for that.
Are you sure about the sequential file is reading the whole XML data as a single record and in a single column?
As mentioned in your first post, if the XML is generated in before job, then tags can be added as follows:In tranformer add the derivattion mentioned in Ray's post. Once you have done this, do not forget to change the XPaths in XML stage to have this root tag.Code: Select all
Seq File --> Transformar--->XML.....(rest of job)
If this thing is not working, pls post the XML generated and some of the columns with their XPaths, then it will be helpful for others to understand.the problem
-
- Participant
- Posts: 437
- Joined: Fri Oct 15, 2004 6:13 am
- Location: Pune, India
I thought there was some way in PX to have the Sequential File stage return only the fully qualified pathname of the file found. Rather than trying to make it do the folder trick of 'reading the entire file into one record' - which while it may work sometimes will never work 100% of the time - see if that can be done.
Then switch the XML Input stage to source using the URL/Filepath option.
Then switch the XML Input stage to source using the URL/Filepath option.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers