DSXchange

Hi All,

how many tokens exist in the address field.

Thanks in Advance

No idea until you explain what you mean by "tokens" and "address".

If I may guess that you mean "how many tokens exist in the address field" then, yes, a Word investigation using an address rule set will yield patterns. The number of characters in the pattern is the number of parsed tokens in the address field.

Yes Ray you are correct. my question is "how many tokens exist in the address field"

That's what I thought you meant, and why I answered that particular question.

To read the entirety of that answer you will need to get yourself a premium membership. Premium membership is one of the ways that the hosting and bandwidth costs of DSXchange (which you will notice carries no advertising) are defrayed.

Have a look at STRIPLIST and SEPLIST in the rule set you are going to use in word investigation. Any character that is in both lists separates tokens in a given string.

mallikharjuna wrote:Yes Ray you are correct. my question is "how many tokens exist in the address field"

JoshGeorge wrote:Have a look at STRIPLIST and SEPLIST in the rule set you are going to use in word investigation. Any character that is in both lists separates tokens in a given string.

mallikharjuna wrote:Yes Ray you are correct. my question is "how many tokens exist in the address field"

As JoshGeorge says, look at the sep and strip lists.
If a character is in the seplist, it will split a token into separate tokens.
If it is in the striplist, it will remove it from the text completely.
If it is in both, it will separate the text and then be removed.

eg:

Original Text: 14/25

Seplist:
Striplist:
Result: 14/25
Tokens: 1

Seplist: /
Striplist:
Result: 14 / 25
Tokens: 3

Seplist:
Striplist: /
Result: 1425
Tokens: 1

Seplist: /
Striplist: /
Result: 14 25
Tokens: 2


Hope this helps.