Page 1 of 2
Unable to view all records from sequential file
Posted: Mon Dec 05, 2011 3:18 am
by mac4rfree85
Hi guys,
I have a sequential file. The file has 2500 rows but when i processed the file using datastage using sequential file, the job is processing only 253 rows.
Even viewdata is showing only 253 rows. No error is occuring.
Can somebody help me to process the whole file.
Cheers!!!
Posted: Mon Dec 05, 2011 5:29 am
by sanjayS
can you able to see all records in Source sequential file using View Data?
Thanks,
sanjay.
Posted: Mon Dec 05, 2011 7:13 am
by TPons
There is option of limiting the number of rows to be processed at run time.
if it has been set, change it to no limit option.
Pons
Posted: Mon Dec 05, 2011 7:39 am
by qt_ky
Check the detailed job log to see if the other rows are getting rejected.
Posted: Mon Dec 05, 2011 7:42 am
by mac4rfree85
Yeah qt_ky. the Other rows are getting rejected saying delim=end is not available. But all the rows have same delimiter then how are those 253 rows are able to process.
Posted: Mon Dec 05, 2011 8:32 am
by chulett
Your metadata doesn't match your file's data, nothing more than that. What settings are you using in the stage? What is different between a row it will process and one it rejects? Can you post an example of each?
Posted: Tue Dec 06, 2011 5:06 am
by mac4rfree85
chulett wrote:Your metadata doesn't match your file's data, nothing more than that. What settings are you using in the stage? What is different between a row it will process and one it rejects? Can you post an example of each?
Following two records got loaded.
Code: Select all
2011-05-03Aveo RETAIL US 12CHEVROLET CARS ENTRY CAR
2011-05-03Aveo FLEET US 250CHEVROLET CARS ENTRY CAR
Following records are not loaded to the table.
Code: Select all
2011-05-03DTS RETAIL US 0CADILLAC CARS LUXURY CAR
2011-05-03DTS FLEET US 59CADILLAC CARS LUXURY CAR
Posted: Tue Dec 06, 2011 5:53 am
by kamtammystuff
What is the metadata in the sequential file stage?
Posted: Tue Dec 06, 2011 5:55 am
by zulfi123786
perform cat -vte <filename> and filter out two loaded records and two records that are not loaded from the above command and list them.
Bascially the stage is unable to find the record delimiter.
Posted: Tue Dec 06, 2011 2:58 pm
by ray.wurlod
Maybe DataStage developers don't like Cadillacs?
Seriously, though, does the 253rd record or thereabouts contain an "end of file" character (Ctrl-D)?
Posted: Tue Dec 06, 2011 11:46 pm
by mac4rfree85
This is what i got
Code: Select all
dsadm@catom-gmapdvl38:/appl/data/Input/US> cat -vte TESTING.txt
2011-05-03DTS RETAIL US 0CADILLAC CARS LUXURY CAR$
2011-05-03DTS FLEET US 59CADILLAC CARS LUXURY CAR$
2011-05-03HHR RETAIL US 7CHEVROLET TRUCKS VAN/MPV$
2011-05-03HHR FLEET US 246CHEVROLET TRUCKS VAN/MPV$
2011-05-03STS RETAIL US 2CADILLAC CARS LUXURY CAR$
2011-05-03Aveo RETAIL US 12CHEVROLET CARS ENTRY CAR$
2011-05-03Aveo FLEET US 250CHEVROLET CARS ENTRY CAR$
Posted: Tue Dec 06, 2011 11:48 pm
by ray.wurlod
Was this file sourced from a Windows system?
Posted: Tue Dec 06, 2011 11:50 pm
by mac4rfree85
Yes ray, i took the file from Windows and ftpped to Unix box, since my datastage is installed in Unix... Also just to answer your question, it is not processing first 253 rows, but random 253 rows from the file.
Posted: Tue Dec 06, 2011 11:56 pm
by ray.wurlod
Could it be that there's a Ctrl-M at the end of the lines (shows as ^M in vi) that is interfering with the parsing? That is, in what mode did you perform the FTP and does your FTP tool handle translating line terminators?
Posted: Wed Dec 07, 2011 12:10 am
by mac4rfree85
That was the first thing i checked. i did not see any Ctrl-M characters..