DSXchange

Posted: **Fri Nov 25, 2005 3:05 pm**

Hi,

We are frequently getting the below fatal error in our team -

main_program: Data set, file set, or file "ProcessFlagCheck:ProcessFlagUAndD.v": Non-pipe (or non-virtual data set) with .v in its name

Since many of us have started getting this error, and all of it seems to have started one fine day, I somehow feel that this is related to the memory in some manner, but I am not sure. But now I am also getting the RT_CONFIG error, which makes my belief stronger that this issue is realted to memory.

I had gone through each stage carefully, according to some of the resolutions that I had seen in this forum, but I am still unable to sort this out. I am also not sure why this is happening.

Can somebody help me out on this.

Regards,

The Bird.

Posted: **Fri Nov 25, 2005 3:10 pm**

Hi,

I forgot to mention the stage where this error was pointed to, in the above post. The "ProcessFlagCheck" stage is a transformer, and "ProcessFlagUAndD" is the output link to an Oracle Enterprise stage.

I am only doing a DateFromDaysSince function in this transformer, and then passing the input fields to the target table.

Any help on this would be appreciated.

regards,

The Bird.

Posted: **Fri Nov 25, 2005 3:57 pm**

You should be able to search for the key words in the error as it has been discussed here before - this being one such example.

Posted: **Fri Nov 25, 2005 6:07 pm**

Craig,

chulett wrote:You should be able to search for the key words in the error as it has been discussed here before - this being one such example.

I did search on this forum, but didn't find any solution to this. I am pasting the generated osh for this below -

Code: Select all

# OSH / orchestrate script for Job CopyOfCopyOfIFilestoESAMediaCatalog compiled at 17:58:42 25 NOV 2005
#################################################################
#### STAGE: AllMediaCatalogSfSrc
## Operator
import
## Operator options
-schema record
  {fill=0, final_delim=end, record_delim='\n', delim=',', null_field='\1', quote=double}
(
  src_appl_cd:string[10];
  ctlg_nme:string[max=100];
  ctlg_yr_num:int32;
  ctlg_ser_cd:string[10];
  ctlg_book_typ_cd:string[10];
  ctlg_win_cd:string[10];
  ctlg_pg_qty:nullable decimal[18,4];
  ctlg_desc:nullable string[max=250];
  ctlg_eff_strt_dt:timestamp;
  ctlg_eff_end_dt:timestamp;
  ctlg_frst_mail_dt:timestamp;
  ctlg_phto_fig_meas:nullable decimal[18,4];
  ctlg_actv_ind:string[1];
  ctlg_prom_ind:string[1];
  ctlg_sea_cd:string[10];
  ctlg_prim_ofr_nme:nullable string[max=100];
  pr_yr_ref_ctlg_nme:nullable string[max=100];
  wrap_prim_ctlg_nme:string[max=100];
  wrap_sec_ctlg_nme:string[max=100];
  media_ctlg_strt_dt:timestamp;
  media_ctlg_end_dt:timestamp;
  src_file_nme:string[max=25];
  src_process_dttm:timestamp;
  process_flag:string[1];
)
-rejects fail
-reportProgress yes
-filepattern '[&SrcFilePath]/[&SrcFileNames]'

## General options
[ident('AllMediaCatalogSfSrc'); jobmon_ident('AllMediaCatalogSfSrc')]
## Outputs
0> [] 'AllMediaCatalogSfSrc:AllMediaCatalogSfSrc.v'
;

#################################################################
#### STAGE: ESASurrKeyNextValDbSrc
## Operator
oraread
## Operator options
-nchar_cs [&DSProjectMapName]
-db_cs [&DSProjectMapName]
-dboptions '{user=[&_ESA_USER_NAME],password=[&_ESA_PASSWORD]}'
-query 'SELECT 
ESA_SURRKEY_LAST_VAL,ESA_TABLE_CD
FROM 
ESA_SURRKEY_NEXTVAL'
-server '[&_ESA_TNS_ENTRY]'
-use_strings

## General options
[ident('ESASurrKeyNextValDbSrc'); jobmon_ident('ESASurrKeyNextValDbSrc')]
## Outputs
0> [modify (
  ESA_SURRKEY_LAST_VAL:nullable decimal[38,10]=ESA_SURRKEY_LAST_VAL;
  ESA_TABLE_CD:not_nullable string[10]=ESA_TABLE_CD;
)] 'ESASurrKeyNextValDbSrc:NextKey.v'
;

#################################################################
#### STAGE: AppendUpdatedSKs
## Operator
transform
## Operator options
-flag run
-name 'V0S32_CopyOfCopyOfIFilestoESAMediaCatalog_AppendUpdatedSKs'

## General options
[ident('AppendUpdatedSKs'); jobmon_ident('AppendUpdatedSKs')]
## Inputs
0< 'SequenceLookupforNextKey:RecsWithNextKey.v'
## Outputs
0> [] 'AppendUpdatedSKs:RecsWithSKs.v'
1> [] 'AppendUpdatedSKs:MediaCatalogLdRecs.v'
;

#################################################################
#### STAGE: UpdateEsaSurrKeyNextValDbTgt
## Operator
oraupsert
## Operator options
-nchar_cs [&DSProjectMapName]
-db_cs [&DSProjectMapName]
-dboptions '{user=[&_ESA_USER_NAME],password=[&_ESA_PASSWORD]}'
-server '[&_ESA_TNS_ENTRY]'
-update 'UPDATE 
[&RefESASurrKeyTable] 
SET 
esa_surrkey_last_val = ORCHESTRATE.esa_surrkey_last_val 
WHERE 
(esa_table_cd = \'MEDCATALOG\')'

## General options
[ident('UpdateEsaSurrKeyNextValDbTgt'); jobmon_ident('UpdateEsaSurrKeyNextValDbTgt')]
## Inputs
0< 'FindMaxSK:MaxSK.v'
;

#################################################################
#### STAGE: MediaCatalogDsTgt
## Operator
copy
## General options
[ident('MediaCatalogDsTgt')]
## Inputs
0< 'AppendUpdatedSKs:MediaCatalogLdRecs.v'
## Outputs
0>| [ds] '[&TgtDatasetPath]/[&TgtDatasetFile]'
;

#################################################################
#### STAGE: ProcessFlagCheck
## Operator
transform
## Operator options
-flag run
-name 'V0S48_CopyOfCopyOfIFilestoESAMediaCatalog_ProcessFlagCheck'

## General options
[ident('ProcessFlagCheck'); jobmon_ident('ProcessFlagCheck')]
## Inputs
0< 'AllMediaCatalogSfSrc:AllMediaCatalogSfSrc.v'
## Outputs
0> [] 'ProcessFlagCheck:ProcessFlagUAndI.v'
1> [] 'ProcessFlagCheck:ProcessFlagUAndD.v'
;

#################################################################
#### STAGE: SequenceLookupforNextKey
## Operator
lookup
## Operator options
-table
-key EsaTblCd
-ifNotFound fail

## General options
[ident('SequenceLookupforNextKey'); jobmon_ident('SequenceLookupforNextKey')]
## Inputs
0< 'ProcessFlagCheck:ProcessFlagUAndI.v'
1< [view (
  EsaTblCd=ESA_TABLE_CD;
)] 'ESASurrKeyNextValDbSrc:NextKey.v'
## Outputs
0> [modify (
keep
  src_appl_cd,ctlg_nme,ctlg_yr_num,ctlg_ser_cd,
  ctlg_book_typ_cd,ctlg_win_cd,ctlg_pg_qty,ctlg_desc,
  ctlg_eff_strt_dt,ctlg_eff_end_dt,ctlg_frst_mail_dt,ctlg_phto_fig_meas,
  ctlg_actv_ind,ctlg_prom_ind,ctlg_sea_cd,ctlg_prim_ofr_nme,
  pr_yr_ref_ctlg_nme,wrap_prim_ctlg_nme,wrap_sec_ctlg_nme,media_ctlg_strt_dt,
  media_ctlg_end_dt,src_file_nme,src_process_dttm,process_flag,
  EsaTblCd,ESA_SURRKEY_LAST_VAL;)] 'SequenceLookupforNextKey:RecsWithNextKey.v'
;

#################################################################
#### STAGE: FindMaxSK
## Operator
tail
## Operator options
-nrecs 10

## General options
[ident('FindMaxSK'); jobmon_ident('FindMaxSK')]
## Inputs
0< 'AppendUpdatedSKs:RecsWithSKs.v'
## Outputs
0> [] 'FindMaxSK:MaxSK.v'
;

#################################################################
#### STAGE: MediaCatalogDbSrc
## Operator
oraupsert
## Operator options
-nchar_cs [&DSProjectMapName]
-db_cs [&DSProjectMapName]
-dboptions '{user=[&_ESA_USER_NAME],password=[&_ESA_PASSWORD]}'
-update 'UPDATE 
[&TgtESATableName] 
SET 
#EsaTblUpdColumn# = ORCHESTRATE.media_ctlg_end_dt1  
WHERE 
(src_appl_cd = ORCHESTRATE.src_appl_cd AND 
ctlg_nme = ORCHESTRATE.ctlg_nme AND 
ctlg_yr_num = ORCHESTRATE.ctlg_yr_num AND 
#EsaTblUpdColumn# = TO_DATE(\'2999-12-31\', \'yyyy-mm-dd\') )'
-reject
-server '[&_ESA_TNS_ENTRY]'

## General options
[ident('MediaCatalogDbSrc'); jobmon_ident('MediaCatalogDbSrc')]
## Inputs
0< 'ProcessFlagCheck:ProcessFlagUAndD.v'
## Outputs
0> [] 'MediaCatalogDbSrc:MediaCatalogDbSrc.v'
;

#################################################################
#### STAGE: RejectOverview
## Operator
peek
## Operator options
-nrecs 10
-name

## General options
[ident('RejectOverview'); jobmon_ident('RejectOverview')]
## Inputs
0< 'RejThreshold:RejThreshold.v'
;

#################################################################
#### STAGE: RejThreshold
## Operator
compare
## Operator options
-abortOnDifference
-field 'RejCount'

## General options
[ident('RejThreshold'); jobmon_ident('RejThreshold'); seq]
## Inputs
0< 'RejectAndError:RejRecCount.v'
1< 'RejectAndError:ChkRejThresh.v'
## Outputs
0> [] 'RejThreshold:RejThreshold.v'
;

#################################################################
#### STAGE: ErrorLogInsDbTgt
## Operator
oraupsert
## Operator options
-nchar_cs [&DSProjectMapName]
-db_cs [&DSProjectMapName]
-dboptions '{user=[&_ESA_USER_NAME],password=[&_ESA_PASSWORD]}'
-update 'UPDATE 
ERROR_PROCESS_LOG 
SET 
SRC_APPL_CD = ORCHESTRATE.SRC_APPL_CD, INTFC_ID = ORCHESTRATE.INTFC_ID, REASON_TXT = ORCHESTRATE.REASON_TXT, REASON_TYP_VAL = ORCHESTRATE.REASON_TYP_VAL, PROCESS_DTTM = ORCHESTRATE.PROCESS_DTTM, FILE_STRING_TXT = ORCHESTRATE.FILE_STRING_TXT 
WHERE 
1=2'
-server '[&_ESA_TNS_ENTRY]'

## General options
[ident('ErrorLogInsDbTgt'); jobmon_ident('ErrorLogInsDbTgt')]
## Inputs
0< 'RejectAndError:ErrorLogInsDbTgt.v'
;

#################################################################
#### STAGE: RejectInsSfTgt
## Operator
export
## Operator options
-schema record
  {final_delim=end, record_delim='\n', delim=',', null_field='\1', quote=double}
(
  src_appl_cd:string[10];
  ctlg_nme:string[max=100];
  ctlg_yr_num:int32;
  ctlg_ser_cd:string[10];
  ctlg_book_typ_cd:string[10];
  ctlg_win_cd:string[10];
  ctlg_pg_qty:nullable decimal[18,4];
  ctlg_desc:nullable string[max=250];
  ctlg_eff_strt_dt:timestamp;
  ctlg_eff_end_dt:timestamp;
  ctlg_frst_mail_dt:timestamp;
  ctlg_phto_fig_meas:nullable decimal[18,4];
  ctlg_actv_ind:string[1];
  ctlg_prom_ind:string[1];
  ctlg_sea_cd:string[10];
  ctlg_prim_ofr_nme:nullable string[max=100];
  pr_yr_ref_ctlg_nme:nullable string[max=100];
  wrap_prim_ctlg_nme:string[max=100];
  wrap_sec_ctlg_nme:string[max=100];
  media_ctlg_strt_dt:timestamp;
  media_ctlg_end_dt:timestamp;
  src_file_nme:string[max=25];
  src_process_dttm:timestamp;
  process_flag:string[1];
)
-file '[&RejectFilePath]/[&RejectFileName]'
-overwrite
-rejects continue

## General options
[ident('RejectInsSfTgt'); jobmon_ident('RejectInsSfTgt')]
## Inputs
0< 'RejectAndError:RejectInsSfTgt.v'
;

#################################################################
#### STAGE: RejectAndError
## Operator
transform
## Operator options
-flag run
-name 'V72S4_CopyOfCopyOfIFilestoESAMediaCatalog_RejectAndError'
-argvalue 'ESAInterfaceId=[&ESAInterfaceId]'
-argvalue 'RejectThreshold=[&RejectThreshold]'

## General options
[ident('RejectAndError'); jobmon_ident('RejectAndError'); seq]
## Inputs
0< 'MediaCatalogDbSrc:MediaCatalogDbSrc.v'
## Outputs
0> [-pp] 'RejectAndError:RejectInsSfTgt.v'
1> [-pp] 'RejectAndError:ErrorLogInsDbTgt.v'
2> [-pp] 'RejectAndError:RejRecCount.v'
3> [-pp] 'RejectAndError:ChkRejThresh.v'
;


# End of OSH code

As this error has been coming quite frequently across multiple jobs and users, I am wondering if this could be an issue within the DataStage server, may be because of some memory issues. I am also getting an RT_CONFIG error sometimes, as mention in the post here -
viewtopic.php?t=96848

This makes me believe more that the issue is not related to the job design.

Regards,

The Bird.

Posted: **Sat Nov 26, 2005 12:16 am**

The RT_CONFIGnnn table contains the runtime version of the job design. I suggest you solve that first, as per the other thread. The error that is the subject of this thread may then magically vanish.

Posted: **Tue Nov 29, 2005 7:04 am**

Thanks Ray!!!

ray.wurlod wrote:The RT_CONFIGnnn table contains the runtime version of the job design. I suggest you solve that first, as per the other thread. The error that is the subject of this thread may then magically vanish.

As you said this error has magically vanished, but I had to delete the current link (on which the error was shown) and replace it with a new link, for this to happen.

And as for the RT_CONFIGnnn table issue, I believe this was indeed a memory issue (as Craig suggested in the other thread). The DataStage Admin ran some utilities on the DS Repository, to sort this out, but I am still not sure if that would solve the problem entirely.

Regards,

The Bird.