reguler expression substitution

memrinal · Post by **memrinal** » Wed Dec 17, 2008 5:21 pm

We have a requirement to strip some bad words from the input - replace the bad words with a '-'.

for bad words, we have a list of reguler expressions - We need to substitute these reguler expressions with a '-'.

Currently I have implemented a demo of it using external filterstage. But I was looking for something within DS which can do this.

Is there a way we can have reguler expressions replaced by another string.

kandyshandy · Post by **kandyshandy** » Wed Dec 17, 2008 5:51 pm

search for "px version of ereplace" in dsxchange

or

Use basic transformer and use ereplace function.

kandyshandy · Post by **kandyshandy** » Wed Dec 17, 2008 5:57 pm

If each bad word is a single character, then use CONVERT function in PX transformer stage.

chulett · Post by **chulett** » Wed Dec 17, 2008 6:08 pm

You mean like 'F' and 'U'?

ray.wurlod · Post by **ray.wurlod** » Wed Dec 17, 2008 6:11 pm

No, those ones would be replaced by something like "#@!&!"

memrinal · Post by **memrinal** » Wed Dec 17, 2008 10:08 pm

Well the reguler expressions need to be replaced by '-' would be like below.

\bDAMN\s?|FART|----(ED|ER|ING|ME|S)\b
\b(HELL|HORNY|J(AC|ER)K-OFF|JI(SM|Z(M)?))\b
\b(MOTHA|MOTHER)?----(ER(S)?|IN(G)?)?\b

I am not familair with ereplace,going to try now.

memrinal · Post by **memrinal** » Thu Dec 18, 2008 6:06 pm

HAs anyone else done regulerexpression repleacement in DS PX? Please let me know.

kandyshandy · Post by **kandyshandy** » Thu Dec 18, 2008 9:00 pm

Use a BASIC transformer. BASIC transformer is available in a PX job. If you want to know more about BASIC transformer, search for it in the forum.

BTW, this is the meaning of regular expression replacement !!

ray.wurlod · Post by **ray.wurlod** » Thu Dec 18, 2008 10:31 pm

Have you considered an External Filter stage that uses something like sed or awk commands/scripts?