Copy stage not optimized out of the job

[zulfi123786]

Hi,

I added a copy stage to the job just to make the job design look appealing and I expected it to be optimized during compilation (Force='FALSE')

The stage has one input link and one output link and all columns just dragged to from input to output but the copy stays in the compiled code and in the score of the job.

Is there an environment variable which overrides the the Force property that I need to modify to have the copy thrown out of the compiled code ?

Below is the osh fragment for Copy stage:

Code: Select all

#### STAGE: Copy_171
## Operator
copy
## General options
[ident('Copy_171'); jobmon_ident('Copy_171')]
## Inputs
0< [] 'seq_WEMED_AETACOEL_Base:lnk_Read.v'
## Outputs
0> [modify (
keep
  MEMBER_ID,EFF_MONTH,PREFIX_NM,LAST_NM,
  FIRST_NM,SUFFIX_NM,MBR_GENDER_CD,BIRTH_DT,
  ADDRESS_LINE_1_TXT,ADDRESS_LINE_2_TXT,CITY_NM,STATE_POSTAL_CD,
  COUNTY_CD,ZIP_CD,MBR_PHONE,SRC_MEMBER_ID,
  MBR_RTP_TYPE_CD,SSN_NBR,PS_UNIQUE_ID,CUSTOMER_NBR,
  GROUP_NBR,SUBGROUP_NBR,ACCOUNT_NBR,SUBS_LAST_NM,
  SUBS_FIRST_NM,SUBS_GENDER_CD,SUBS_BIRTH_DT,SUBS_ZIP_CD,
  SUBS_ST_POSTAL_CD,SUBS_COUNTY_CD,SUBS_SSN_NBR,FILE_ID,
  PLSP_PROD_CD,PRODUCT_LN_CD,FUND_CTG_CD,COVERAGE_TYPE_CD,
  NTWK_SRV_AREA_ID,CUST_SEGMENT_CD,MED_IND,DRUG_IND,
  SA_IND,MH_IND,ORIG_EFF_DT,PCP_TAX_ID_NBR,
  PCP_PRVDR_ID,PCP_NAME,PCP_ADDRESS_LINE_1_TXT,PCP_ADDRESS_LINE_2_TXT,
  PCP_CITY_NM,PCP_STATE_POSTAL_CD,PCP_ZIP_CD,PCP_NPI_NBR,
  PCP_SPECIALTY_CTG_CD,ORG_ID,FIRST_VST_DT,LAST_VST_DT,
  VISIT_NBR,ATTR_TAX_ID_NBR,ATTR_PRVDR_ID,ATTR_NAME,
  ATTR_ADDRESS_LINE_1_TXT,ATTR_ADDRESS_LINE_2_TXT,ATTR_CITY_NM,ATTR_STATE_POSTAL_CD,
  ATTR_ZIP_CD,ATTR_NPI_NBR,ATTR_SPECIALTY_CTG_CD,ATTR_STATUS_IND,
  ATTR_EFF_DT,ATTR_CNCL_DT,SRC_MEMBER_ID_EMP,SRC_CUMB_ID_MBR,
  SRC_CUMB_ID_EMP,MIDDLE_NM,INDIVIDUAL_ID,PULSE_FSI_SCORE,
  ERG_PROSP_SCORE,ERG_RETRO_SCORE,BUSINESS_LN_CD,AETNA_CARD_ID,
  GRP_CNTL_NAME,CURRENT_REC_INDIC,FILE_DATE;
)] 'Copy_171:lnk_OldLayout.v'
;

Thanks

[jerome_rajan]

Try running the job and check the job score (set APT_DUMP_SCORE environment variable) to see if the copy operator appears there

[zulfi123786]

Try running the job and analyze the check the job score (set APT_DUMP_SCORE environment variable) to see if the copy operator appears there

as mentioned in my original post the stage stays in the score of the job.

but the copy stays in the compiled code and in the score of the job.

[jerome_rajan]

FYI, the Copy Stage will not get optimized out during compilation but only after the score is composed which is post job run.

Now that you mention that you ran the job and the score still contains the copy operator, remember that by setting the option to 'false' you are giving the optimizer an option and not asking it to definitively optimize the copy out. Perhaps, in your case, the optimizer decided that it could do with the copy stage.

[ray.wurlod]

Can you show us the copy operator in the score?