Server Funnel

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
ds2000
Premium Member
Premium Member
Posts: 109
Joined: Sun Apr 22, 2007 7:25 pm
Location: ny

Server Funnel

Post by ds2000 »

How Parallel Funnel stage functionality can be used in server job. Want to union two sequential files in a server job. Please comment...thanx.
tcj
Premium Member
Premium Member
Posts: 98
Joined: Tue Sep 07, 2004 6:57 pm
Location: QLD, Australia
Contact:

Post by tcj »

Should be able to use the link collector. Have two sequential links coming into the link collector with a single output from the link collector.

The two sequential links must have the same metadata.

This will merge two sequential files into the one sequential file with the same metadata.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

This can cause problems if one file has more rows than the other. The Link Collector waits for the never-to-arrive next row, and eventually takes a timeout error.

Better is to use a filter command in your sequential file stage that creates a stream of all lines from the two files. I prefer TYPE as the command; others like COPY.

Code: Select all

TYPE file1 file2

Code: Select all

COPY file1 + file2
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
jib
Participant
Posts: 4
Joined: Thu Jan 09, 2003 4:37 pm
Location: India

Re: Server Funnel

Post by jib »

Use OS command to append one file with the other if there is no header in files. type File1 >> File2 or cat File1 >> File2
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Depending on what 'union' means to you, you can also investigate the Merge stage.
-craig

"You can never have too many knives" -- Logan Nine Fingers
TD_Chile
Premium Member
Premium Member
Posts: 10
Joined: Tue Jun 10, 2008 1:43 pm
Location: Santiago, Chile

Post by TD_Chile »

Ray, how could the link collector cause problems? I use it all the time and sometimes with 10 differents archives (with the same metadata off course) and always works fine.
The CAT, COPY and TYPE commands are also good options too.
TD_Chile
Premium Member
Premium Member
Posts: 10
Joined: Tue Jun 10, 2008 1:43 pm
Location: Santiago, Chile

Post by TD_Chile »

By the way, if you are joining only two files you can also append the files using 2 sequential file stages and in the input from the second specified the option "append to existing file"
TD_Chile
Premium Member
Premium Member
Posts: 10
Joined: Tue Jun 10, 2008 1:43 pm
Location: Santiago, Chile

Post by TD_Chile »

By the way, if you are joining only two files you can also append the files using 2 sequential file stages and in the input from the second specified the option "append to existing file"
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:? Not if both links are 'firing' at the same time. Sequential files by their very nature support multiple readers but only one writer at a time, so you'd need to arrange for them to run in a serial fashion.
-craig

"You can never have too many knives" -- Logan Nine Fingers
TD_Chile
Premium Member
Premium Member
Posts: 10
Joined: Tue Jun 10, 2008 1:43 pm
Location: Santiago, Chile

Post by TD_Chile »

sorry, I forgot to specified that the sequential files must be one after the other to do the append
Post Reply