rownum for a record

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
India2000
Participant
Posts: 274
Joined: Sun Aug 22, 2010 11:07 am

rownum for a record

Post by India2000 »

Hi,

I have scenario to select records from a flat file using rownumber.

In the transformer if I use the constraint to select rec>10 I get more than 10 .

How can I get the exact number of records from a file with exact row numbers attached to them.

Rownumber property is used to attach rowninber to records in the flat file.

Can anyone pleasgive any hint to get the desired result?

Thank you
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

To 'get the exact number of records from a file' you would need to count them before the job runs. Typically. And off the top of my head you can leverage the @INROWNUM system variable to know what row number you are working with, as long as you are careful with your partitioning. I.E. don't partition. Perhaps even just use a Server job as doing this as any kind of a parallel operation seems like total overkill. At least from what little you've mentioned so far.
-craig

"You can never have too many knives" -- Logan Nine Fingers
India2000
Participant
Posts: 274
Joined: Sun Aug 22, 2010 11:07 am

Post by India2000 »

Hi Chullet,

I have the count. I want to select only a specific number of records from the file.



Thank you,
Last edited by India2000 on Tue Oct 11, 2016 7:17 pm, edited 1 time in total.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

In the Available Properties to Add pane for Options in the Sequential File stage you will find a property called Read First Rows. This takes an integer value, which can be a job property.

Alternately use a Filter command such as head -10 in the Sequential File stage or emplace a Head, Tail or Sample stage immediately after the Sequential File stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
naveenkumar.ssn
Participant
Posts: 36
Joined: Thu Dec 03, 2009 9:11 pm
Location: Malaysia

Re: rownum for a record

Post by naveenkumar.ssn »

hi

Do you mean to say you have a rownum column in a sequential file and you wanted to take the first 10 records from the file..if so you can specify head -10 {FileName.txt} ..It will fetch you the first 10 records..However if you dont have a rownum column and wanted to take the first 10 rows from the file ..use the function INROWNUM and get the first 10 records.

Regards
Naveen
Naveen Kumar
Datastage Consultant
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Not at all.

The Read First Rows property, or the head command, will return the first N rows irrespective of the data in the file.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

As is normally the situation, there are multiple solutions to this requirement. Since you are on UNIX, you may be able to only bring in the records desired or you may need to filter what you need out in the job.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply