reference link from a shared container

sumitgulati · Post by **sumitgulati** » Thu Jan 15, 2004 4:01 pm

Hi,

I have a shared container say 'X' which I am using in my server job. In the server job I am passing few columns to the shared container 'X' and the remaining columns to a transformer 'T'. I want the output from container 'X' and transformer 'T' to be joined using another transformer say 'T1' and then put them into a table.
I am not able to acheive this because neither from transformer 'T' nor from the container 'X' I can draw a reference link to transformer 'T1'.

One way to solve this is to put the output from 'X' into a hash file and then draw a reference link from this hash file to 'T1'. I want to avoid using hash file or any sequential file or a table. Is there any way I can do it.

Thanks in advance

Regards,
Sumit

vmcburney · Post by **vmcburney** » Thu Jan 15, 2004 4:25 pm

Lookup data needs to be stored in a table that supports a key lookup, which is why they tend to be hash files or database tables. In order for your shared container rows to be available as a lookup you need to put them into a hash file or database table.

It will be faster if you can avoid having to split and rejoin your data by either sending all your fields through the container or moving the container logic into the job.

sumitgulati · Post by **sumitgulati** » Thu Jan 15, 2004 5:04 pm

Hi Vincent,

The shared container is being used in colse to 50 different server Jobs having diffirent number and type of columns. Because of this its not possible to pass all the columns through the container.
Is there any way in which I can pass all the columns through the container. That would definitely give better performance as the rejoining will be avoided.

Regards,
Sumit

vmcburney · Post by **vmcburney** » Thu Jan 15, 2004 6:40 pm

It's tricky. One option is to take the transform logic out of the shared container and put it into one or more routines. Then your fifty or so jobs all call these routines within a transformer to do the same transformation your shared container would have done. It really depends on what your container is trying to do! It looks like it is creating more work for you as you have to go through the effort of splitting and merging fields.

The other option is a bit messy, it involves packing your extra columns into a single delimited text field, passing it to the shared container as a single long text field, then uppacking it again when it arrives back in the job.

I think the hash file lookup or the shared routines are better options.

sumitgulati · Post by **sumitgulati** » Thu Jan 15, 2004 6:47 pm

Thanks Vincent,

I will try to work on the options you gave.

Regards,
Sumit

ray.wurlod · Post by **ray.wurlod** » Thu Jan 15, 2004 8:36 pm

sumitgulati wrote:Hi Vincent,

The shared container is being used in colse to 50 different server Jobs having diffirent number and type of columns. Because of this its not possible to pass all the columns through the container.
Is there any way in which I can pass all the columns through the container. That would definitely give better performance as the rejoining will be avoided.

Regards,
Sumit

Presumably the number of other columns varies, too.
One possibility would be to pass ONE extra column through the container, containing a delimited string of all the other columns, then to decompose this again in T1.