sequential:50 out of 200

mcs@rajesh · Post by **mcs@rajesh** » Sun Oct 14, 2007 9:36 pm

Hello Everybody,
I have a paticular job in that i need to send only first 50 rows of records out of 200 from the source plugin to the other plugins can any one tell a simplest way to do this...

ray.wurlod · Post by **ray.wurlod** » Sun Oct 14, 2007 10:43 pm

If it's in sequential mode use a Filter command that invokes the head command. Otherwise use a downstream Head stage.

mcs@rajesh · Post by **mcs@rajesh** » Tue Oct 16, 2007 8:50 pm

ray.wurlod wrote:If it's in sequential mode use a Filter command that invokes the head command. Otherwise use a downstream Head stage. ...

Thanks For ur reply, i tried with this but am not getting correctly..can you walk through the steps..

ray.wurlod · Post by **ray.wurlod** » Wed Oct 17, 2007 9:16 am

SequentialFile ----> Head ----> other stages

What - precisely - did you try?

Ramani · Post by **Ramani** » Fri Oct 19, 2007 2:28 am

I think there should be something to do with the partition for this. Because the propery Number of rows to copy from input to output is per partition.

So what is the partition type here. Is it "Entire"

Thanks
Ramani.

mcs@rajesh · Post by **mcs@rajesh** » Fri Oct 19, 2007 6:28 am

ray.wurlod wrote:SequentialFile ----> Head ----> other stages

What - precisely - did you try? ...

Based upon my business rules i need pass first 50 out of 200...

mcs@rajesh · Post by **mcs@rajesh** » Fri Oct 19, 2007 6:30 am

Ramani wrote:I think there should be something to do with the partition for this. Because the propery Number of rows to copy from input to output is per partition.

So what is the partition type here. Is it "Entire"

Thanks
Ramani.

Thanks for ur reply the partition used here is round robin.. I dont think Entire partition will work for this job...

stefanfrost1 · Post by **stefanfrost1** » Fri Oct 19, 2007 7:26 am

If I am not misunderstanding this entirely, you want to read the 50 first rows from a sequncial file!?!

So using a sequncial file stage (in sequncial mode with one reader) and a filter with the specification head -50, will only provide you with one node containg the first 50 records of your file. If you later decide to partition this data for some processing thats fine and you will only be processing those 50 rows spread across your configuration.

You could also use external source stage with the command head -50 <filename> for the same effect (remember not to read the data using more nodes because that will probably give you 50 records per node)

balajisr · Post by **balajisr** » Sun Oct 21, 2007 11:13 pm

If all you need is to read first 50 rows from the sequential file then Set "Read First Rows" property in sequential file stage to 50.