Run Time Column Propogation Issue.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kapil008
Participant
Posts: 20
Joined: Wed Sep 14, 2005 1:49 pm

Run Time Column Propogation Issue.

Post by kapil008 »

In my Job design I am using run time column propogation. The job design is simple...But it uses "Run Time Column Propogation"

Job design : Table_A ---> Transformer ----> Table_B

The metadata of the tables A and B are

DLV_DT VARCHAR2(8 BYTE),
FRQ VARCHAR2(3 BYTE),
PCS_DT VARCHAR2(8 BYTE),

My question is that when I run the job, it seems that DS imports the metadata and implicitly ensures that all VarChar type are Unicode. This is causing my job to break. I changed the job design to import the metadata without using Unicode and it worked.

However, If i use Run time column propagation, it seems that it imports all the VarChar metadata fields in Unicode. Is there a way to disable it??

Thanks,

Kapil
rasi
Participant
Posts: 464
Joined: Fri Oct 25, 2002 1:33 am
Location: Australia, Sydney

Post by rasi »

Hi

You can disable this by using Datastage Administrator. Logon to the project where you want to disable. Inside that untick the Enable Runtime Column Propagation for Parallel Jobs

Regards
Regards
Siva

Listening to the Learned

"The most precious wealth is the wealth acquired by the ear Indeed, of all wealth that wealth is the crown." - Thirukural By Thiruvalluvar
kwwilliams
Participant
Posts: 437
Joined: Fri Oct 21, 2005 10:00 pm

Re: Run Time Column Propogation Issue.

Post by kwwilliams »

You can also turn it off in the job properties if you want to leave it turned on in the administrator. I personally would not disable it in the admin, but I would not have it default to RCP on. It just causes too many headaches.
ameyvaidya
Charter Member
Charter Member
Posts: 166
Joined: Wed Mar 16, 2005 6:52 am
Location: Mumbai, India

Post by ameyvaidya »

From what I remember from the Manuals:

RCP is a feature of the base orchestrate framework. It cannot be switched "off". What actually happens when RCP is switched off is an implicit Modify operator is added after every stage that changes column definitions. This may mean potentially doubling the number of operators in your job causing performance hits.

Ideally RCP should be switched off at certain stages in your job where leaving it on will cause all those headaches(Defaulting, dropping issues). Example of such a stage would be funnel.

A recommended best practice is to document through annotations where in the job RCP has been switched off.
Amey Vaidya<i>
I am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand.</i>
<i>- Douglas Adams</i>
kapil008
Participant
Posts: 20
Joined: Wed Sep 14, 2005 1:49 pm

Post by kapil008 »

rasi wrote:Hi

You can disable this by using Datastage Administrator. Logon to the project where you want to disable. Inside that untick the Enable Runtime Column Propagation for Parallel Jobs

Regards
Hi I think I was not that clear. What I ment is that in my Job design I want run time column propogation. However, what I want to disable is the during run time the metadata that it imports for VarChar is in Unicode format. That is causing the problem.

Any ideas how this could be achieved :roll:
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi,
How about, if your mannually take off the unicode option for varchar field. You can also try to remove from dsx as a whole if the above doesnt works. Out of curiosity, what makes your job to break if unicode present?

-Kumar
kapil008
Participant
Posts: 20
Joined: Wed Sep 14, 2005 1:49 pm

Post by kapil008 »

kumar_s wrote:Hi,
How about, if your mannually take off the unicode option for varchar field. You can also try to remove from dsx as a whole if the above doesnt works. Out of curiosity, what makes your job to break if unicode present?

-Kumar
Kumar,

We since I am using Run Time Column propogation and not manually defining it, I am not sure how to disable Unicode. If I manually define the metadata and remove Unicode it works. But in my Job design it is required that i use Run time column propogation

kapil
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Still you have the option to edit dsx :idea: (may be not advisable by everyone :wink: )

-Kumar
kapil008
Participant
Posts: 20
Joined: Wed Sep 14, 2005 1:49 pm

Post by kapil008 »

kumar_s wrote:Still you have the option to edit dsx :idea: (may be not advisable by everyone :wink: )

-Kumar
Hmm...and how do we remove it from dsx... :?:
aramachandra
Participant
Posts: 55
Joined: Tue Sep 20, 2005 10:58 am

Post by aramachandra »

What is the version of your source database. Not sure you can turn it off through datastage....

.i.e with RCP it will dependon your source database system, and how it interprets the meta data...

I am interested in knowing myself how to turn it off if one can....

arvind
Post Reply