Joiner, union etc

Stupe · Post by **Stupe** » Wed Aug 29, 2007 4:28 am

Hi all,

I'm very new to Datastage, have just completed installation on my local machine (7.5x2) and am now messing around with it.

I come from an Informatica background, and well, DS's GUI confuses me. I successfully managed to do simple loads (DB table>flat file; DB table>simple transformation>flat file) using Server jobs. But I didn't notice any objects that are frequently found and used in Informatica, ie Joiner, Union, Sorter, Filter.

However I did notice that more objects are available if a Parallel job is created.

How does one do the similar activities in Datastage, like in Informatica?

mohdsuf · Post by **mohdsuf** » Wed Aug 29, 2007 4:48 am

Yes you can all you want to do in Datastage as you were doing in Informatica. Various activities of joiner, sort, filter etc you can observe and use it very well in parallel jobs from the PROCESSING palette.
Just check it out and you will have fun using datastage.

Stupe · Post by **Stupe** » Wed Aug 29, 2007 4:54 am

Thanks Sufiyan.

Do correct me if I'm wrong, but isn't a parallel job restricted to unix systems? What if I just want to perform the functions with a Server job?

mohdsuf · Post by **mohdsuf** » Wed Aug 29, 2007 5:08 am

You cant say that Parallel jobs are restricted only to UNIX systems, but considering the stability and security what UNIX provides it certainly has an edge. As far as functions you want to build and use in your server jobs you can also do it with the help of subroutines(written in BASIC) to be used in your jobs.

Stupe · Post by **Stupe** » Wed Aug 29, 2007 5:23 am

Ah... so, if I do continue using Server jobs, I'll have to code my subroutines.

And these transformations will be available in Parallel jobs. Can I run Parallel jobs on my local machine (containing Source, Datastage, Target)?

mohdsuf · Post by **mohdsuf** » Wed Aug 29, 2007 5:28 am

I think Datastage 7.5X2 supports running parallel jobs on windows. And in the transformer(parallel) you can use various inbuilt system variables, functions, type conversion and various others. And if you feel that these are less then also you can build your routines in C/C++ for parallel jobs. Understand the difference BASIC for server jobs routines and C/C++ for parallel jobs routines.

mohdsuf · Post by **mohdsuf** » Wed Aug 29, 2007 5:29 am

I think Datastage 7.5X2 supports running parallel jobs on windows. And in the transformer(parallel) you can use various inbuilt system variables, functions, type conversion and various others. And if you feel that these are less then also you can build your routines in C/C++ for parallel jobs. Understand the difference BASIC for server jobs routines and C/C++ for parallel jobs routines.

VCInDSX · Post by **VCInDSX** » Wed Aug 29, 2007 9:08 am

Stupe wrote:Ah... so, if I do continue using Server jobs, I'll have to code my subroutines.

And these transformations will be available in Parallel jobs. Can I run Parallel jobs on my local machine (containing Source, Datastage, Target)?

Well, as long as you have Enterprise Edition Server and you can create parallel jobs you don't have to worry about whether you are on Windows or UNIX. However, based on your server's hardware configuration you will gain by utilizing the Parallel Engine's capabilities.

Also, you design the jobs using the Designer and compile them. Execution happens on the server. So you are not running the jobs on your "local machine" per se.... unless you have the client and server on the same box and you are using the client on the server box to develop.

I am sure other Gurus can throw more light on the subject.... This is my understanding...

Good luck

ray.wurlod · Post by **ray.wurlod** » Wed Aug 29, 2007 3:57 pm

You have version 7.5x2, which is parallel-capable.
The main thing is that the design process is a very different mindset. In Informatica you're used to a bottom-up design strategy - start with the details and build the structure (maps and the like) on those. In DataStage it's top-down - start with the big picture (essentially the picture you draw in Designer) and fill in the details subsequently.

Stupe · Post by **Stupe** » Thu Aug 30, 2007 12:24 am

Thanks for the replies guys, but unfortunately can't see the whole of Ray's comments.

But just thinking out loud, I assume the licensing structure for Parallel-capable and a non parallel-capable is different. One of the added benefits of parallel-capable will definitely be being able to split the load to different processors. I get that.

But it looks like the non parallel-capable version is very much inferior, since 'simple' transformation objects like joiners, filters, sorts etc needs some manual coding (can't say how difficult to code though, never/haven't tried it).

girish.rupkumar · Post by **girish.rupkumar** » Wed Sep 26, 2007 4:02 am

Hi...

In Server 7.5x2, if you are using OCI, RDBMS or any other Database stages, you can join two tables using SQL query. If you are using sequential file or any flat file, you can do all the join types except right outer join by using a transformer stage.

Coming to Sorting, there is a sort stage. Again you can use this stage when your source is sequential or flat files. If the data is coming from database, you can sort the data in OCI stage or any other datbase stages.

DSXchange

Joiner, union etc

Joiner, union etc

Re: Joiner, union etc