Classifying Telephone numbers in a lookup

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Classifying Telephone numbers in a lookup

Post by admin »

Hello to all.
Ive got a problem, i wonder if any one can be of assistance, id be really
grateful.
Im trying to classify a list of telephone numbers, according to a cod thats
stored in a table with the list of all telephone prefix, my problem is that
not all prefixes have the same length so ive got to find a way to do a
dynamic lookup,the problem is that my key is depending on the prefix thats
stored on the lookup table.
Can anybody point me the way?

Thank you for your patient:
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

What kind of database is your lookup stored in? Is it a relational database or a hash file?

Can you give more detail about what your lookup table looks like and what you are trying to get from it?

-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: Tuesday, 30 October 2001 8:21 PM
To: datastage-users@oliver.com
Subject: Classifying Telephone numbers in a lookup

Hello to all.
Ive got a problem, i wonder if any one can be of assistance, id be really grateful. Im trying to classify a list of telephone numbers, according to a cod thats stored in a table with the list of all telephone prefix, my problem is that not all prefixes have the same length so ive got to find a way to do a dynamic lookup,the problem is that my key is depending on the prefix thats stored on the lookup table. Can anybody point me the way?

Thank you for your patient:
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Well my database is oracle-8i and ive got the result of a extraction that
has a column named telephone and has values like 219999999, 91111111111,
112, 0800234234234, 112123456789, ...
and my table which has the prefix and the code has two columns:
prefix_code: 0001, 0002, 1230, ...
prefix: 21, 0800, 112, ...
And ive got to match the beginning of the telephone number (prefix) with is
prefix_code, im trying to do this by using a transformer stage in which i do
a look up using the prefix as key but ive got to transform the telephone in
to his prefix in order to find a match,
the problem as you can see is that neither the telephone or the prefix has a
fixed length.


>From: "David Barham"
>Reply-To:
>To:
>Subject: RE: Classifying Telephone numbers in a lookup
>Date: Tue, 30 Oct 2001 20:39:08 +1000
>
>What kind of database is your lookup stored in? Is it a relational
>database
>or a hash file?
>
>Can you give more detail about what your lookup table looks like and
>what you are trying to get from it?
>
>-----Original Message-----
>From: pedro santos [mailto:pedro_f_santos@hotmail.com]
>Sent: Tuesday, 30 October 2001 8:21 PM
>To: datastage-users@oliver.com
>Subject: Classifying Telephone numbers in a lookup
>
>Hello to all.
>Ive got a problem, i wonder if any one can be of assistance, id be
>really grateful. Im trying to classify a list of telephone numbers,
>according to a cod thats
>stored in a table with the list of all telephone prefix, my problem is that
>not all prefixes have the same length so ive got to find a way to do a
>dynamic lookup,the problem is that my key is depending on the prefix thats
>stored on the lookup table.
>Can anybody point me the way?
>
>Thank you for your patient:
>Pedro
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at
>http://explorer.msn.com/intl.asp
>


_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Can do!


* Make sure the DataStage definition of the prefix on the lookup is long enough to hold the entire telephone number.
* Pass the entire telephone number to the lookup in the key column "prefix"
* Write a user defined query for the lookup something like:

select prefix, prefix_code
from table_name
where :1 like prefix||%

and this should return you the row(s) where the prefix matches the telephone number.


Complication. Depending on your data, it is foreseeable that this could return more than one row. I will make the rash assumption that the more precisely defined prefix is the preferred one. If this is the case, you could extend the query as follows:

select * from (
select prefix, prefix_code
from table_name
where :1 like prefix||%
order by length(prefix) desc
) where rownum = 1

For the benefit of other readers, please note that this construct assumes features of Oracle 8i and will not work in 8.0 or earlier.

I have not tested this, although I have done similar things to achieve fuzzy data matching. If it does not work precisely as I have typed it, it should at least get you started.

Have fun.

David



-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: Tuesday, 30 October 2001 8:54 PM
To: datastage-users@oliver.com
Subject: RE: Classifying Telephone numbers in a lookup

Well my database is oracle-8i and ive got the result of a extraction that has a column named telephone and has values like 219999999, 91111111111, 112, 0800234234234, 112123456789, ... and my table which has the prefix and the code has two columns:
prefix_code: 0001, 0002, 1230, ...
prefix: 21, 0800, 112, ...
And ive got to match the beginning of the telephone number (prefix) with is prefix_code, im trying to do this by using a transformer stage in which i do a look up using the prefix as key but ive got to transform the telephone in to his prefix in order to find a match, the problem as you can see is that neither the telephone or the prefix has a fixed length.


>From: "David Barham"
>Reply-To:
>To:
>Subject: RE: Classifying Telephone numbers in a lookup
>Date: Tue, 30 Oct 2001 20:39:08 +1000
>
>What kind of database is your lookup stored in? Is it a relational
>database or a hash file?
>
>Can you give more detail about what your lookup table looks like and
>what you are trying to get from it?
>
>-----Original Message-----
>From: pedro santos [mailto:pedro_f_santos@hotmail.com]
>Sent: Tuesday, 30 October 2001 8:21 PM
>To: datastage-users@oliver.com
>Subject: Classifying Telephone numbers in a lookup
>
>Hello to all.
>Ive got a problem, i wonder if any one can be of assistance, id be
>really grateful. Im trying to classify a list of telephone numbers,
>according to a cod thats
>stored in a table with the list of all telephone prefix, my problem is that
>not all prefixes have the same length so ive got to find a way to do a
>dynamic lookup,the problem is that my key is depending on the prefix thats
>stored on the lookup table.
>Can anybody point me the way?
>
>Thank you for your patient:
>Pedro
>
>_________________________________________________________________
>Get your FREE download of MSN Explorer at
>http://explorer.msn.com/intl.asp
>


_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Thank you David i will try that.


>From: "David Barham"
>Reply-To:
>To:
>Subject: RE: Classifying Telephone numbers in a lookup
>Date: Tue, 30 Oct 2001 21:14:09 +1000
>
>Can do!
>
>
>* Make sure the DataStage definition of the prefix on the lookup is
>long enough to hold the entire telephone number.
>* Pass the entire telephone number to the lookup in the key column
>"prefix"
>* Write a user defined query for the lookup something like:
>
>select prefix, prefix_code
>from table_name
>where :1 like prefix||%
>
>and this should return you the row(s) where the prefix matches the
>telephone
>number.
>
>
>Complication. Depending on your data, it is foreseeable that this
>could return more than one row. I will make the rash assumption that
>the more precisely defined prefix is the preferred one. If this is the
>case, you could extend the query as follows:
>
>select * from (
>select prefix, prefix_code
>from table_name
>where :1 like prefix||%
>order by length(prefix) desc
>) where rownum = 1
>
>For the benefit of other readers, please note that this construct
>assumes features of Oracle 8i and will not work in 8.0 or earlier.
>
>I have not tested this, although I have done similar things to achieve
>fuzzy
>data matching. If it does not work precisely as I have typed it, it should
>at least get you started.
>
>Have fun.
>
>David
>
>
>
>-----Original Message-----
>From: pedro santos [mailto:pedro_f_santos@hotmail.com]
>Sent: Tuesday, 30 October 2001 8:54 PM
>To: datastage-users@oliver.com
>Subject: RE: Classifying Telephone numbers in a lookup
>
>Well my database is oracle-8i and ive got the result of a extraction
>that has a column named telephone and has values like 219999999,
>91111111111, 112, 0800234234234, 112123456789, ... and my table which
>has the prefix and the code has two columns:
>prefix_code: 0001, 0002, 1230, ...
>prefix: 21, 0800, 112, ...
>And ive got to match the beginning of the telephone number (prefix)
>with
>is
>prefix_code, im trying to do this by using a transformer stage in which i
>do
>a look up using the prefix as key but ive got to transform the telephone
>in
>to his prefix in order to find a match,
>the problem as you can see is that neither the telephone or the prefix has
>a
>fixed length.
>
>
> >From: "David Barham"
> >Reply-To:
> >To:
> >Subject: RE: Classifying Telephone numbers in a lookup
> >Date: Tue, 30 Oct 2001 20:39:08 +1000
> >
> >What kind of database is your lookup stored in? Is it a relational
> >database or a hash file?
> >
> >Can you give more detail about what your lookup table looks like and
> >what you are trying to get from it?
> >
> >-----Original Message-----
> >From: pedro santos [mailto:pedro_f_santos@hotmail.com]
> >Sent: Tuesday, 30 October 2001 8:21 PM
> >To: datastage-users@oliver.com
> >Subject: Classifying Telephone numbers in a lookup
> >
> >Hello to all.
> >Ive got a problem, i wonder if any one can be of assistance, id be
>really
> >grateful.
> >Im trying to classify a list of telephone numbers, according to a cod
> >thats stored in a table with the list of all telephone prefix, my
> >problem is
>that
> >not all prefixes have the same length so ive got to find a way to do
> >a dynamic lookup,the problem is that my key is depending on the
> >prefix
>thats
> >stored on the lookup table.
> >Can anybody point me the way?
> >
> >for your patient:
> >Pedro
> >
> >_________________________________________________________________



_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Pedro,

were done this before in data stage, doing dialcode lookups against worldwide dial codes. This is quite a hard thing to do, because as you said, dialcodes differ between countries (on style and length) on a dialled number you have no idea what part or how much of it is the number and how much is the dialcode.

The solution we used made use of b-tree format hashfiles, as was quick. The exercise of actually doing the lookup is quite simple, and we ended up with a routine that looked into the prebuilt hash tables. depending on the number of dialcode lookups in the hash, even a normal hash file could be slow (just down to the complexities of the amount of lookup and kind of guess work that the lookup needs to do).

This was specific consultancy work, so maybe email your account manager at Ascential and cc myself, and well see if a similar project could be set up for you.

Andrew Webb
Principal Presales Consultant
Ascential Software
Changing the Way the World Looks At Information www.ascentialsoftware.com

Mobile : +44 (0)777 5762347
Direct : +44 (0)20 8818 0717
Switchboard : +44 (0)20 8818 0700
Fax: +44 (0)20 8818 0701

Email: andrew.webb@ascentialsoftware.com

This message is intended for the sole use of the intended recipient and may contain information that is confidential and privileged and/or attorneys work product. Any review or distribution by any other person is prohibited. If you are not the intended recipient, please immediately contact the sender and delete all copies




-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: 30 October 2001 10:21 AM
To: datastage-users@oliver.com
Subject: Classifying Telephone numbers in a lookup


Hello to all.
Ive got a problem, i wonder if any one can be of assistance, id be really
grateful.
Im trying to classify a list of telephone numbers, according to a cod thats

stored in a table with the list of all telephone prefix, my problem is that
not all prefixes have the same length so ive got to find a way to do a
dynamic lookup,the problem is that my key is depending on the prefix thats
stored on the lookup table.
Can anybody point me the way?

Thank you for your patient:
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

Post by admin »

Pedro,

I tried using Type 25 type files for lookups for this reason before and got a core dump. I will try again.

If you want some consulting then we may be able to help also. We have a number of DataStage consultants available in New Zealand and if the logistics can be worked out then the NZD is cheaper than the USD.

Phil.

-----Original Message-----
From: andrew.webb@ascentialsoftware.com [mailto:andrew.webb@ascentialsoftware.com]
Sent: Wednesday, October 31, 2001 1:02 AM
To: datastage-users@oliver.com
Cc: pedro_f_santos@hotmail.com
Subject: RE: Classifying Telephone numbers in a lookup


Pedro,

were done this before in data stage, doing dialcode lookups against worldwide dial codes. This is quite a hard thing to do, because as you said, dialcodes differ between countries (on style and length) on a dialled number you have no idea what part or how much of it is the number and how much is the dialcode.

The solution we used made use of b-tree format hashfiles, as was quick. The exercise of actually doing the lookup is quite simple, and we ended up with a routine that looked into the prebuilt hash tables. depending on the number of dialcode lookups in the hash, even a normal hash file could be slow (just down to the complexities of the amount of lookup and kind of guess work that the lookup needs to do).

This was specific consultancy work, so maybe email your account manager at Ascential and cc myself, and well see if a similar project could be set up for you.

Andrew Webb
Principal Presales Consultant
Ascential Software
Changing the Way the World Looks At Information www.ascentialsoftware.com

Mobile : +44 (0)777 5762347
Direct : +44 (0)20 8818 0717
Switchboard : +44 (0)20 8818 0700
Fax: +44 (0)20 8818 0701

Email: andrew.webb@ascentialsoftware.com

This message is intended for the sole use of the intended recipient and may contain information that is confidential and privileged and/or attorneys work product. Any review or distribution by any other person is prohibited. If you are not the intended recipient, please immediately contact the sender and delete all copies




-----Original Message-----
From: pedro santos [mailto:pedro_f_santos@hotmail.com]
Sent: 30 October 2001 10:21 AM
To: datastage-users@oliver.com
Subject: Classifying Telephone numbers in a lookup


Hello to all.
Ive got a problem, i wonder if any one can be of assistance, id be really grateful. Im trying to classify a list of telephone numbers, according to a cod thats

stored in a table with the list of all telephone prefix, my problem is that not all prefixes have the same length so ive got to find a way to do a dynamic lookup,the problem is that my key is depending on the prefix thats stored on the lookup table. Can anybody point me the way?

Thank you for your patient:
Pedro

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
Locked