Generating records in ascending sequence

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kumarjit
Participant
Posts: 99
Joined: Fri Oct 12, 2012 7:47 am
Location: Kolkata

Generating records in ascending sequence

Post by kumarjit »

Hi.

I have a source file with a single input column in it.
A sample input might look like:

Code: Select all

Col1
1
8
4
5
7
3
10
My requirement is to get a single column output(say Col2), with record values ranging between min(col1) and max(col1), and in a ascending order

Code: Select all

Col2
1
2
3
4
5
6
7
8
9
10
One approach that I implemented was:
Step 1: Get the max and min of col1
Step 2: Do a sparse lookup on an Oracle table with following query to generate the ascending record values:
SELECT R FROM ( SELECT LEVEL AS R FROM DUAL CONNECT BY LEVEL<=Orchestrate.max(col1)) WHERE R>=Orchestrate.min(col1)

But, I am seeking an alternative approach, something that does not employ a database interaction at all.

Can I be achieved?
Please suggest.

Warm Regards,
Kumarjit.
Pain is the best teacher, but very few attend his class..
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Transformer looping.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Am I missing something, or is this just about sorting?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

It needs to be sorted and then the gaps filled in.
-craig

"You can never have too many knives" -- Logan Nine Fingers
kumarjit
Participant
Posts: 99
Joined: Fri Oct 12, 2012 7:47 am
Location: Kolkata

Post by kumarjit »

@Ray: As Craig has mentioned, the output would be a sorted column(col2), values of which will be continuous and range from min(col1) to max(col1).

@Craig: Looping does work good in this case where the input is already sorted in ascending order, but what if the input column values are staggered(like below), and I do not want to use any sort stage/inline sort feature to sort the column.

Code: Select all

Col1 
8
10
1
5
3
4
10
7
Any help?

Regards.
Pain is the best teacher, but very few attend his class..
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You will need to employ some kind of sorting, even if you are maintaining the sorted list in the Transformer loop.

It would be much easier to sort the source data so that your job can know the start and end points of the values that need to be interpolated.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
anbu
Premium Member
Premium Member
Posts: 596
Joined: Sat Feb 18, 2006 2:25 am
Location: india

Post by anbu »

Input --> Aggregator --> Transformer --> Output

- Get Min and Max from aggregator
- Use Max in Transformer looping and generate all the values
You are the creator of your destiny - Swami Vivekananda
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

kumarjit wrote:I do not want to use any sort stage/inline sort feature to sort the column.
Why the heck not? :?

As noted, you can ignore what was actually sent and just generate a sequence from min to max.
-craig

"You can never have too many knives" -- Logan Nine Fingers
ShaneMuir
Premium Member
Premium Member
Posts: 508
Joined: Tue Jun 15, 2004 5:00 am
Location: London

Post by ShaneMuir »

kumarjit wrote:My requirement is to get a single column output(say Col2), with record values ranging between min(col1) and max(col1), and in a ascending order
kumarjit wrote: I do not want to use any sort stage/inline sort feature to sort the column.
How would you ever hope to output values in ascending order on the output without using some form of sort?
If you input order is important to you in any way - just add a column which maintains the input order (eg use @INROWNUM) and then sort it again after you've calculated the missing numbers - though considering your output is just a sequence of numbers this is all quite a ridiculous requirement?

Or you've left a lot of requirements out.
kumarjit
Participant
Posts: 99
Joined: Fri Oct 12, 2012 7:47 am
Location: Kolkata

Post by kumarjit »

Yes, anbu's solution worked, though the requirement was indeed ridiculous. :oops:

Regards,
Kumarjit.
Pain is the best teacher, but very few attend his class..
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

<rant>Resist stupid requirements!</rant>
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply