Delimiter for field warning while reading sequential file

[ketanshah123]

HI

I am getting many warnings while running my job for seq. file stage

Measure_code,0: Delimiter for field "ZZPA_M001_RKF008" not found; input: {G _ A V}, at offset: 153 [impexp/group_comp.C:5775]

I have checked in my file delimiter is present for field value.

Any suggestion how to remove these warnings


Thanx in advance

[ray.wurlod]

What is your specified delimiter. The stage has read the four characters indicated. Could there be line terminators in your data?

[gssr]

HI

I am getting many warnings while running my job for seq. file stage

Measure_code,0: Delimiter for field "ZZPA_M001_RKF008" not found; input: {G _ A V}, at offset: 153 [impexp/group_comp.C:5775]

I have checked in my file delimiter is present for field value.

Any suggestion how to remove these warnings


Thanx in advance

Did any records get rejected??

[ketanshah123]

Delimiter definedis Comma , and no rows are getting rejected

My file has total 97 fields and first row is field name and same field name are values for that column .. for eg :

G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100
G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100

1st row is name and 2nd row is values for that field.

[agpt]

Delimiter definedis Comma , and no rows are getting rejected

My file has total 97 fields and first row is field name and same field name are values for that column .. for eg :

G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100
G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100

1st row is name and 2nd row is values for that field.

Is it same for all the records in your data? and if so, I really am not able to understand use of this data.
Well, are you getting some output or nothing at all? there is a possibility that at some value comma is missing ?

[rohithmuthyala]

If there are any columns read as Char try changing them to Varchar and check the result.

[Althaf6553]

Can you run cat -vte <file name> on Unix
Check if you are getting any ^ M or Junk Chars in the data .
A wrong file transfer might have caused these

[ArndW]

Also, what is your data type definition for field "ZZPA_M001_RKF008"

[ketanshah123]

All field are of varchar datatype. I am also getting correct output nothing is rejected but this warning messages I need to take care of.

[ArndW]

Varchar datatype, but what width does that column have?

[ketanshah123]

Can you run cat -vte <file name> on Unix
Check if you are getting any ^ M or Junk Chars in the data .
A wrong file transfer might have caused these

after running above command there are $ signs coming in :


G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100,G_AVV450,ZZPA_M001_RKF002,ZZPA_M001_RKF003,ZZPA_M001_RKF004,ZZPA_M001_RKF005,ZZPA_M001_RKF006,ZZPA_M001_RKF007,ZZPA_M001_$
RKF008,ZZPA_M001_RKF009,ZZPA_M001_RKF010,ZZPA_M001_RKF012,ZZPA_M001_RKF011,ZZPA_M001_RKF014,ZZPA_M001_RKF015,ZZPA_M001_RKF016,G_AVV140,G_AVV145,G_AVV150,G_AV$
V155,G_AVV160,G_AVV165,G_AVV170,G_AVV330,G_AVV175,G_AVV180,G_AVV190,ZZPA_M001_RKF017,ZZPA_M001_RKF018,ZZPA_M001_RKF019,ZZPA_M001_RKF020,ZZPA_M001_RKF021,ZZPA$
_M001_RKF022,ZZPA_M001_RKF023,ZZPA_M001_RKF024,ZZPA_M001_RKF025,ZZPA_M001_RKF026,ZZPA_M001_RKF027,ZZPA_M001_RKF028,ZZPA_M001_RKF029,ZZPA_M001_RKF030,ZZPA_M00$
1_RKF211,ZZPA_M001_RKF032,ZZPA_M001_RKF033,ZZPA_M001_RKF034,ZZPA_M001_RKF035,ZZPA_M001_RKF036,ZZPA_M001_RKF037,ZZPA_M001_RKF040,ZZPA_M001_RKF041,ZZPA_M001_RK$
F042,ZZPA_M001_RKF043,ZZPA_M001_RKF044,ZZPA_M001_RKF045,ZZPA_M001_RKF046,ZZPA_M001_RKF047,ZZPA_M001_RKF048,ZZPA_M001_RKF049,ZZPA_M001_RKF050,ZZPA_M001_RKF051$
,ZZPA_M001_RKF052,ZZPA_M001_RKF053,ZZPA_M001_RKF054,ZZPA_M001_RKF055,G_AVV240,G_AVV340,ZZPA_M001_CKF056R,ZZPA_M001_CKF057R,ZZPA_M001_CKF058R,ZZPA_M001_CKF059$
R,ZZPA_M002_RKF095,ZZPA_M002_RKF097,ZZPA_M002_RKF096,G_AVV110,G_AVV255,G_AVV256,G_AVV257,G_AVV258,G_AVV259,G_AVV260,G_AVV310,G_AVV345,G_AVV350,G_AVV365,G_AVV$
395,G_AVV415,ZZPA_M002_RKF222,ZZPA_M002_RKF223,ZZPA_M002_RKF224,ZZPA_M002_RKF225,ZZPA_M002_RKF227,G_AVV425,G_AVV445,ZZPA_M002_RKF180$
0G_QVVLBS,0G_QVVIQT,0G_QVVALT,0G_AVV100,0G_AVV450,ZZPA_M001_RKF002,ZZPA_M001_RKF003,ZZPA_M001_RKF004,ZZPA_M001_RKF005,ZZPA_M001_RKF006,ZZPA_M001_RKF007,ZZPA_M001_RKF008,ZZPA_M001_RKF009,ZZPA_M001_RKF010,ZZPA_M001_RKF012,ZZPA_M001_RKF011,ZZPA_M001_RKF014,ZZPA_M001_RKF015,ZZPA_M001_RKF016,0G_AVV140,0G_AVV145,0G_AVV150,0G_AVV155,0G_AVV160,0G_AVV165,0G_AVV170,0G_AVV330,0G_AVV175,0G_AVV180,0G_AVV190,ZZPA_M001_RKF017,ZZPA_M001_RKF018,ZZPA_M001_RKF019,ZZPA_M001_RKF020,ZZPA_M001_RKF021,ZZPA_M001_RKF022,ZZPA_M001_RKF023,ZZPA_M001_RKF024,ZZPA_M001_RKF025,ZZPA_M001_RKF026,ZZPA_M001_RKF027,ZZPA_M001_RKF028,ZZPA_M001_RKF029,ZZPA_M001_RKF030,ZZPA_M001_RKF211,ZZPA_M001_RKF032,ZZPA_M001_RKF033,ZZPA_M001_RKF034,ZZPA_M001_RKF035,ZZPA_M001_RKF036,ZZPA_M001_RKF037,ZZPA_M001_RKF040,ZZPA_M001_RKF041,ZZPA_M001_RKF042,ZZPA_M001_RKF043,ZZPA_M001_RKF044,ZZPA_M001_RKF045,ZZPA_M001_RKF046,ZZPA_M001_RKF047,ZZPA_M001_RKF048,ZZPA_M001_RKF049,ZZPA_M001_RKF050,ZZPA_M001_RKF051 ,ZZPA_M001_RKF052,ZZPA_M001_RKF053,ZZPA_M001_RKF054,ZZPA_M001_RKF055,0G_AVV240,0G_AVV340,ZZPA_M001_CKF056R,ZZPA_M001_CKF057R,ZZPA_M001_CKF058R,ZZPA_M001_CKF059R,ZZPA_M002_RKF095,ZZPA_M002_RKF097,ZZPA_M002_RKF096,0G_AVV110,0G_AVV255,0G_AVV256,0G_AVV257,0G_AVV258,0G_AVV259,0G_AVV260,0G_AVV310,0G_AVV345,0G_AVV350,0G_AVV365,0G_AVV395,0G_AVV415,ZZPA_M002_RKF222,ZZPA_M002_RKF223,ZZPA_M002_RKF224,ZZPA_M002_RKF225,ZZPA_M002_RKF227,0G_AVV425,0G_AVV445,ZZPA_M002_RKF180$

[ketanshah123]

Can you run cat -vte <file name> on Unix
Check if you are getting any ^ M or Junk Chars in the data .
A wrong file transfer might have caused these

after running above command there are $ signs coming in :


G_QVVLBS,G_QVVIQT,G_QVVALT,G_AVV100,G_AVV450,ZZPA_M001_RKF002,ZZPA_M001_RKF003,ZZPA_M001_RKF004,ZZPA_M001_RKF005,ZZPA_M001_RKF006,ZZPA_M001_RKF007,ZZPA_M001_$
RKF008,ZZPA_M001_RKF009,ZZPA_M001_RKF010,ZZPA_M001_RKF012,ZZPA_M001_RKF011,ZZPA_M001_RKF014,ZZPA_M001_RKF015,ZZPA_M001_RKF016,G_AVV140,G_AVV145,G_AVV150,G_AV$
V155,G_AVV160,G_AVV165,G_AVV170,G_AVV330,G_AVV175,G_AVV180,G_AVV190,ZZPA_M001_RKF017,ZZPA_M001_RKF018,ZZPA_M001_RKF019,ZZPA_M001_RKF020,ZZPA_M001_RKF021,ZZPA$
_M001_RKF022,ZZPA_M001_RKF023,ZZPA_M001_RKF024,ZZPA_M001_RKF025,ZZPA_M001_RKF026,ZZPA_M001_RKF027,ZZPA_M001_RKF028,ZZPA_M001_RKF029,ZZPA_M001_RKF030,ZZPA_M00$
1_RKF211,ZZPA_M001_RKF032,ZZPA_M001_RKF033,ZZPA_M001_RKF034,ZZPA_M001_RKF035,ZZPA_M001_RKF036,ZZPA_M001_RKF037,ZZPA_M001_RKF040,ZZPA_M001_RKF041,ZZPA_M001_RK$
F042,ZZPA_M001_RKF043,ZZPA_M001_RKF044,ZZPA_M001_RKF045,ZZPA_M001_RKF046,ZZPA_M001_RKF047,ZZPA_M001_RKF048,ZZPA_M001_RKF049,ZZPA_M001_RKF050,ZZPA_M001_RKF051$
,ZZPA_M001_RKF052,ZZPA_M001_RKF053,ZZPA_M001_RKF054,ZZPA_M001_RKF055,G_AVV240,G_AVV340,ZZPA_M001_CKF056R,ZZPA_M001_CKF057R,ZZPA_M001_CKF058R,ZZPA_M001_CKF059$
R,ZZPA_M002_RKF095,ZZPA_M002_RKF097,ZZPA_M002_RKF096,G_AVV110,G_AVV255,G_AVV256,G_AVV257,G_AVV258,G_AVV259,G_AVV260,G_AVV310,G_AVV345,G_AVV350,G_AVV365,G_AVV$
395,G_AVV415,ZZPA_M002_RKF222,ZZPA_M002_RKF223,ZZPA_M002_RKF224,ZZPA_M002_RKF225,ZZPA_M002_RKF227,G_AVV425,G_AVV445,ZZPA_M002_RKF180$
0G_QVVLBS,0G_QVVIQT,0G_QVVALT,0G_AVV100,0G_AVV450,ZZPA_M001_RKF002,ZZPA_M001_RKF003,ZZPA_M001_RKF004,ZZPA_M001_RKF005,ZZPA_M001_RKF006,ZZPA_M001_RKF007,ZZPA_M001_RKF008,ZZPA_M001_RKF009,ZZPA_M001_RKF010,ZZPA_M001_RKF012,ZZPA_M001_RKF011,ZZPA_M001_RKF014,ZZPA_M001_RKF015,ZZPA_M001_RKF016,0G_AVV140,0G_AVV145,0G_AVV150,0G_AVV155,0G_AVV160,0G_AVV165,0G_AVV170,0G_AVV330,0G_AVV175,0G_AVV180,0G_AVV190,ZZPA_M001_RKF017,ZZPA_M001_RKF018,ZZPA_M001_RKF019,ZZPA_M001_RKF020,ZZPA_M001_RKF021,ZZPA_M001_RKF022,ZZPA_M001_RKF023,ZZPA_M001_RKF024,ZZPA_M001_RKF025,ZZPA_M001_RKF026,ZZPA_M001_RKF027,ZZPA_M001_RKF028,ZZPA_M001_RKF029,ZZPA_M001_RKF030,ZZPA_M001_RKF211,ZZPA_M001_RKF032,ZZPA_M001_RKF033,ZZPA_M001_RKF034,ZZPA_M001_RKF035,ZZPA_M001_RKF036,ZZPA_M001_RKF037,ZZPA_M001_RKF040,ZZPA_M001_RKF041,ZZPA_M001_RKF042,ZZPA_M001_RKF043,ZZPA_M001_RKF044,ZZPA_M001_RKF045,ZZPA_M001_RKF046,ZZPA_M001_RKF047,ZZPA_M001_RKF048,ZZPA_M001_RKF049,ZZPA_M001_RKF050,ZZPA_M001_RKF051 ,ZZPA_M001_RKF052,ZZPA_M001_RKF053,ZZPA_M001_RKF054,ZZPA_M001_RKF055,0G_AVV240,0G_AVV340,ZZPA_M001_CKF056R,ZZPA_M001_CKF057R,ZZPA_M001_CKF058R,ZZPA_M001_CKF059R,ZZPA_M002_RKF095,ZZPA_M002_RKF097,ZZPA_M002_RKF096,0G_AVV110,0G_AVV255,0G_AVV256,0G_AVV257,0G_AVV258,0G_AVV259,0G_AVV260,0G_AVV310,0G_AVV345,0G_AVV350,0G_AVV365,0G_AVV395,0G_AVV415,ZZPA_M002_RKF222,ZZPA_M002_RKF223,ZZPA_M002_RKF224,ZZPA_M002_RKF225,ZZPA_M002_RKF227,0G_AVV425,0G_AVV445,ZZPA_M002_RKF180$

[ketanshah123]

Varchar datatype, but what width does that column have? ...

width of all column is 20

[ArndW]

Those "$" signs might be coming from your terminal emulator, since the line breaks are continued on the next line. How about using "vi" to see if the data is correct? If the first line is column names, then look at line 2 in vi and at position 153, does that correspond to the error message?

[ray.wurlod]

The problem is at the 153rd character position. That should make it very easy to track down, using a utility such as a hex editor (or even an od -xc command).