Hi,
I have a simple job which reads from a file and updates a particular field in a oracle database.
The file has around 20 mil record and the commit count is 10000. The partition type is Auto(havent changed anything).
Now the job aborted in the middle due to oracle instance being down. It had updated around 12 mil records.
Now if we re-run this again all the 20 mil records would be taken for update. But since the partition is Auto, we are not sure whether the updated records are the first 12 mil records from the file which we can remove and run this job again with just remaining 8 mil records.
Can anyone comment on this.
partition question
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If you are reading the file in sequential mode, then you can skip the first 12 million rows (or whatever the number is) with impunity.
If you are reading the file in parallel mode (perhaps more than one reader per node), then it's still possible, but you need to check how many rows have been processed on each node (from Monitor, perhaps, or DSGetLinkInfo() with the "partition row count" option). You may need to back off a little, say to 10 million just to be safe.
If you are reading the file in parallel mode (perhaps more than one reader per node), then it's still possible, but you need to check how many rows have been processed on each node (from Monitor, perhaps, or DSGetLinkInfo() with the "partition row count" option). You may need to back off a little, say to 10 million just to be safe.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
7.1 at a guess, maybe even a minor 7.0 release.
Obviously it's more difficult with delimited data, but it's certainly possible (locate the percentage point then scan forward for a line terminator).
Obviously it's more difficult with delimited data, but it's certainly possible (locate the percentage point then scan forward for a line terminator).
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
I tried it at 8.0 and it won't.ray.wurlod wrote:Not true any more. Multiple readers per node will work with delimited formats.
Sequential_File_33: The multinode option requires fixed length records.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>