Hi All,
My job design is as below,
External source stage ---> XML Input stage (url input) ---> Data set
I have set of XML files in a folder. With the help of external source stage i am reading the file urls and passing it as source for XML input stage. I need the out from XML input stage for all files into one data set. So that i can use one single job to read all data for further processing. so i have selected the append operation in data set.
It is incremental load. So before the next day's process i want the dataset to be cleared. Since data set in append mode i will be having old data also in the dataset now.
Please let me know how can i have an empty dataset before every day's processing.
TIA.
Append operation for DataSet - output from xml input stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 39
- Joined: Tue Feb 28, 2012 12:59 pm
- Location: Chennai
Try it. Using this technique, it will work without append mode. When you have (say) 300 xml files in a single directory, a single Job run will read thru all of them, delivering parsed rows to your DataSet .....as though it was one big xml document.
Ernie
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Re: Append operation for DataSet - output from xml input sta
As noted, that is exactly what will happen by default, no need to append or clear - just replace each time.techiexavier wrote: need the out from XML input stage for all files into one data set.
And ps - you are not using a Server job. I corrected your post and moved it to the proper forum.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 39
- Joined: Tue Feb 28, 2012 12:59 pm
- Location: Chennai