Input record sequence number
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 342
- Joined: Tue Nov 04, 2008 10:38 am
- Location: Chennai, India
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
Yes Row number starts with zero. But How do you get different partition in sequential file?arunkumarmm wrote:I just tried this by keeping a sequential file as a source. But the sequence number starts with 0 for every partition.anbu wrote:If your input is sequential file, then you can set Row Number column option in Properties to get the sequential number.
You are the creator of your destiny - Swami Vivekananda
Does your input file have a header record or column headers in the first sequential record? That might be why your maximum row number is greater than the number of data records. That's a guess, because when the documentation says "column header records are ignored during processing" it doesn't go into detail for this situation.
Your count and actual matches only if you have equal number of rows in all the partitionsarunkumarmm wrote:Thanks Vinoth! But no luck with this as well. When I use this, The sequence number starts with 1 but gives a count of 177. Actual is 175. I will try with some other set of records and let you know.vinothkumar wrote:Can you try with this
( (@INROWNUM - 1) * @NUMPARTITIONS + @PARTITIONNUM + 1)
You are the creator of your destiny - Swami Vivekananda
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
I'm not sure about this. But I have 4 partitions and I get 4 different sequence numbers starting from 0anbu wrote:Yes Row number starts with zero. But How do you get different partition in sequential file?arunkumarmm wrote:I just tried this by keeping a sequential file as a source. But the sequence number starts with 0 for every partition.anbu wrote:If your input is sequential file, then you can set Row Number column option in Properties to get the sequential number.
Arun
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
I just have a test file created by me. It doesnt have the header or a trailer.FranklinE wrote:Does your input file have a header record or column headers in the first sequential record? That might be why your maximum row number is greater than the number of data records. That's a guess, because when the documentation says "column header records are ignored during processing" it doesn't go into detail for this situation.
Arun
-
- Participant
- Posts: 246
- Joined: Mon Jun 30, 2008 3:22 am
- Location: New York
- Contact:
Yes. I got it now. This is the reason! Thanks Anbu. But is there a way to over come this?anbu wrote:Your count and actual matches only if you have equal number of rows in all the partitionsarunkumarmm wrote:Thanks Vinoth! But no luck with this as well. When I use this, The sequence number starts with 1 but gives a count of 177. Actual is 175. I will try with some other set of records and let you know.vinothkumar wrote:Can you try with this
( (@INROWNUM - 1) * @NUMPARTITIONS + @PARTITIONNUM + 1)
Arun
Seriously?Run the parallel transformer in sequential way and use @INROWNUM
Mike Hester
mhester@petra-ps.com
mhester@petra-ps.com
Agreed, but the answer should have clarified that point. There are many times that a single node configuration is preferable to a multi node, or running an operator in the sequential mode, but without knowing a bit more from the op the answer needs to be qualified.
Mike Hester
mhester@petra-ps.com
mhester@petra-ps.com