How to create and execute routines in PX

[Marley777]

Hi, I have created code in unix scripts then called the scripts from a sequencer, but would like to learn more about creating and calling DS routines. Can someone point me to a good tutorial of some kind? Also, why would I need a routine instead of a unix shell script(looking for an example)?

[asorrell]

This is what is called a "non-trivial topic". I will address the "why", but for the "how" you need to read these documents:

Chapter 5 of the Parallel Job Advanced Developers Guide located in your installation documentation (note: not available via download from IBM) and the "guaranteed to put an insomniac to sleep in 10 minutes" Custom Operators Reference Guide http://publibfp.dhe.ibm.com/epubs/pdf/c1929250.pdf.

WHY you'd want to use a routine is simple - you want to add custom functionality to DataStage that can be run in parallel to avoid slowing down a parallel processing job stream with a single-threaded procedure.

Fortunately there are multiple ways to accomplish this including build-ops, custom stages and wrappers (all of which are covered in Chapter 5).

Originally many existing custom stages and operators were written by IBM consultants that knew C++ better than the DataStage transformer, or felt it would be faster to run custom C++. Lately, with the transformer speed and functionality improvements in recent releases, I have been asked by many clients to replace these custom stages with transformers. This is because they don't have the expertise to support the custom operators, which have become "black boxes" that can't be upgraded or fixed by their in-house staff.

[vinothkumar]

Try this link also

http://it.toolbox.com/blogs/dw-soa/data ... easy-20926

[Marley777]

Thanks for the response so far.

also wondering if I can use datastage basic code to create a routine that will be called in a px job? Or do I have to use C++?

[anbu]

BASIC transformer is available in PX job. But it will be very slow.

You have to use C++ to create parallel routines

[chulett]

Unless we are talking about before/after routines, those are still in BASIC even for Parallel jobs.

[Marley777]

If we have custom routines written for server jobs and want to convert to px jobs, how can we do that? Do we need to rewrite in C++?

Thanks.

[datisaq]

Try this link too..
http://issaq-infomania.blogspot.com/

[ArndW]

If we have custom routines written for server jobs and want to convert to px jobs, how can we do that? Do we need to rewrite in C++?

Thanks.

Yes, you will need to recode these routines to c++ for PX jobs. While you can use BASIC transform stages in PX jobs and call your original routines from these stages, you lose performance and some functionality when doing so.

[asorrell]

If we have custom routines written for server jobs and want to convert to px jobs, how can we do that? Do we need to rewrite in C++?

Maybe - my experience has been that many custom routines can be recreated using standard PX stages combined with wrapped UNIX commands, eliminating the need for C++.

[Marley777]

Hi, is this statement true

"The parallel routines can only be called from the parallel transformer(datastage parallel job transformer) in a datastage job."

Was thinking you can execute routines in a sequencer?

[anbu]

Yes parallel routines can be called only from the Parallel Transformer.

You cannot call parallel routines from the Sequencer.

[kduke]

Andy

How fast are UNIX commands compared to compiled C routine?