How to count records in unix

chrisjones · Post by **chrisjones** » Sun Jul 23, 2006 4:41 pm

Hi ,

In XML i am having records, every record is separated by delilmiter(@@#),so based on this delimiter i want to count the number of records.

so can you help me with syntax how to count records..

chulett · Post by **chulett** » Sun Jul 23, 2006 5:33 pm

I don't think I've ever seen XML with delimiters like that, but regardless...

Clarify exactly what you've got. Is it one long 'record' that you need to break apart around this three-character delimiter or something else? If you do a 'wc -l' on the file, does it return a 1?

I don't recall any built in function that can handle a delimiter like that. You may need to build something like an awk script - or a C program - to read the file and handle the records based on those delimiters. How are you parsing this now?

What's your ultimate goal with all of these 'in UNIX' questions? Perhaps if you said what you were attempting to do at a more strategic (not tactical) level you could get some better advice on how to approach it...

chrisjones · Post by **chrisjones** » Sun Jul 23, 2006 7:35 pm

Thanks Hullet for your quick reply,
Actually i am generating an xml file by combining DAT Files using unix script ,in that single xml file i am having multple records separated by delimiter(@@#),so i want to count the number of records in that xml file in order to compare the records in the xml file with Flag file records in order to make sure that we receive the correct records from source..

Thanks,
Chris

i aim to

chulett wrote:I don't think I've ever seen XML with delimiters like that, but regardless...

Clarify exactly what you've got. Is it one long 'record' that you need to break apart around this three-character delimiter or something else? If you do a 'wc -l' on the file, does it return a 1?

I don't recall any built in function that can handle a delimiter like that. You may need to build something like an awk script - or a C program - to read the file and handle the records based on those delimiters. How are you parsing this now?

What's your ultimate goal with all of these 'in UNIX' questions? Perhaps if you said what you were attempting to do at a more strategic (not tactical) level you could get some better advice on how to approach it...

chulett · Post by **chulett** » Sun Jul 23, 2006 8:07 pm

Clear as mud. How does one 'generate an xml file by combining DAT files'?

Perhaps it would help if you posted examples of these oddly delimited records.

DSguru2B · Post by **DSguru2B** » Sun Jul 23, 2006 10:20 pm

A piece of advice. These questions are more appropriate for a Unix forum rather than DataStage. I am not saying we wont help you, dont get me wrong, what i am trying to imply here is that you will get answers more quickly or even find them by googling it much faster.
Regards,

kumar_s · Post by **kumar_s** » Mon Jul 24, 2006 4:37 am

Why only in unix and not in datastage?

ameyvaidya · Post by **ameyvaidya** » Mon Jul 24, 2006 5:35 am

with the limited detail available this is what I'd try out:

If the XML File has only one delimiter present per line:
grep -c Displays only a count of matching lines.
for More information check "man grep" in Unix.

Using DataStage:
Read in the entire file (Folder Stage). Use the DCount() function. Search the BASIC guide for more details.

DSguru2B · Post by **DSguru2B** » Mon Jul 24, 2006 12:47 pm

grep -c will only return 1 if there are no line terminators. The OP has still not confirmed whether is a single huge line or multiple lines. For that he needs to perform a sed operation to replace the martian delimiter with a LF and then simply do a wc -l. That should do it.
Something like

Code: Select all

sed 's/@@#/\n/g' yourFile.xml | wc -l

clshore · Post by **clshore** » Mon Jul 24, 2006 1:16 pm

Or,
try nawk:

nawk -F'@@#' '{print NF}' yourFile.xml

The -F specifies a Regular Expression for field seperator, the NF is the number of fields.

I don't think '@' or '#' are ERE metacharacters, but if so, just backslash escape them.

Carter