Limits for row procesing

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
akrzy
Participant
Posts: 121
Joined: Wed Dec 08, 2004 4:46 am

Limits for row procesing

Post by akrzy »

Can I limit amount of rows when I run the job?

The "Limits" tab in the Director isn't available.
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Either use a user with access to the LIMITS (developers) or put in a parameter and program logic to limit the number of rows processed.
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Unfortunately row limits are only available to server jobs, not parallel jobs. There is no easy way to do it in parallel jobs.

The sequential file stage has an optional option called "Read First Rows" and you can set this and pass a number in as a job parameter. There is no easy way to turn it off other then removing the option and recompiling the job.

Database stages may support a row limit in a where clause, such as a row number statement in Oracle or a TOP statement in SQL Server. Once again they may be difficult to dynamically turn on and off.
xlnc
Participant
Posts: 7
Joined: Mon Feb 21, 2005 4:43 pm

Post by xlnc »

what is your input file as, if its a database file then you can surely limit the rows according to your requirement... if your imput is any of the databases then use these following querys to limit the rows and if your input is a dataset or a sequential file then use a transformer and RUN it in sequential mode later on go to constraints and use @inrownum<'your desired number of rows'...that should work.

-DB2 select * from table fetch first 10 rows only
-Informix select first 10 * from table
-Microsoft SQL Server and Access select top 10 * from table
-MySQL and PostgreSQL select * from table limit 10
-Oracle 8i select * from (select * from table) where rownum <= 10
-Teradata : select * from table sample 10

10x
akrzy
Participant
Posts: 121
Joined: Wed Dec 08, 2004 4:46 am

Post by akrzy »

Thank a lot for your promts.
If you have any other idea please tell me :)

Regards,
Anka
nkreddy
Premium Member
Premium Member
Posts: 23
Joined: Mon Jun 21, 2004 7:12 am
Location: New York

Post by nkreddy »

But the queries retrieve random rows and I am not sure if they pick the first ten...incase if you want the first ten.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

As long as you do not want the data ordered by and not go too much indepth with the row migration, they must be much accurate. Anyhow it must be fine to allow a sample rows flow through.
akrzy
Participant
Posts: 121
Joined: Wed Dec 08, 2004 4:46 am

Post by akrzy »

I don't need retrive the first rows.

I just want to limit the source rows.
xlnc
Participant
Posts: 7
Joined: Mon Feb 21, 2005 4:43 pm

Post by xlnc »

Well, Firstly whats your requirement? then whats your input ?as I said If your source is a database then do a group by and use this funtion then you will get the first 10 rows, and if your souce is any seq file then you have to necessarily use a Xformer and run in sequencial mode so that you can stop the inrows to what ever your req is....
XLNC
xlnc
Participant
Posts: 7
Joined: Mon Feb 21, 2005 4:43 pm

Post by xlnc »

My Bad.... I mean ORDER BY :(
XLNC
Post Reply