473,287 Members | 1,708 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Regular expression: non-latin word/non-word characters and UTF-8

Hi

I wrote a function that "normalizes" strings for use in URLs in a UTF-8
encoded content administration application. After having removed the accents
from latin characters I try to remove all non-word characters from the
string:

// PCRE syntax:
$string = preg_replace("/([\W]+)/", "-", $string);

// POSIX alternative (mb_string is on):
$string = ereg_replace("[^[:alnum:]]+", "-", $string);

// post-process and return
return urlencode(trim($string, "-"));

Both ways work but remove all non-latin characters. But what I want to do is
remove only the non-word characters of whatever languages, and keep all word
characters regardless if they are Japanese, Hebrew, Arab, Latin or whatever.

Is there a way for a Regex to recognize non-latin word/non-word characters?
Or do I have to manually specify all the characters to be removed?

Thanks for every hint
Markus
Sep 22 '05 #1
1 6859
i suppose your *other* post didn't supply you with an agreeable answer and
that *re-posting* the same question will. well, guess what? i doubt
re-posting it will find you an any more agreeable donor.

....but that's just me thinking out loud.
"Markus Ernst" <derernst@NO#SP#AMgmx.ch> wrote in message
news:43**********@news.cybercity.ch...
| Hi
|
| I wrote a function that "normalizes" strings for use in URLs in a UTF-8
| encoded content administration application. After having removed the
accents
| from latin characters I try to remove all non-word characters from the
| string:
|
| // PCRE syntax:
| $string = preg_replace("/([\W]+)/", "-", $string);
|
| // POSIX alternative (mb_string is on):
| $string = ereg_replace("[^[:alnum:]]+", "-", $string);
|
| // post-process and return
| return urlencode(trim($string, "-"));
|
| Both ways work but remove all non-latin characters. But what I want to do
is
| remove only the non-word characters of whatever languages, and keep all
word
| characters regardless if they are Japanese, Hebrew, Arab, Latin or
whatever.
|
| Is there a way for a Regex to recognize non-latin word/non-word
characters?
| Or do I have to manually specify all the characters to be removed?
|
| Thanks for every hint
| Markus
|
|
Sep 22 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Bradley Plett | last post by:
I'm hopeless at regular expressions (I just don't use them often enough to gain/maintain knowledge), but I need one now and am looking for help. I need to parse through a document to find a URL,...
9
by: MJ | last post by:
HI I want to know what is mean by regular expression in C Mayur
18
by: Q. John Chen | last post by:
I have Vidation Controls First One: Simple exluce certain special characters: say no a or b or c in the string: * Second One: I required date be entered in "MM/DD/YYYY" format: //+4 How...
2
by: Sehboo | last post by:
Hi, I have several regular expressions that I need to run against documents. Is it possible to combine several expressions in one expression in Regex object. So that it is faster, or will I...
3
by: James D. Marshall | last post by:
The issue at hand, I believe is my comprehension of using regular expression, specially to assist in replacing the expression with other text. using regular expression (\s*) my understanding is...
7
by: Billa | last post by:
Hi, I am replaceing a big string using different regular expressions (see some example at the end of the message). The problem is whenever I apply a "replace" it makes a new copy of string and I...
9
by: Pete Davis | last post by:
I'm using regular expressions to extract some data and some links from some web pages. I download the page and then I want to get a list of certain links. For building regular expressions, I use...
25
by: Mike | last post by:
I have a regular expression (^(.+)(?=\s*).*\1 ) that results in matches. I would like to get what the actual regular expression is. In other words, when I apply ^(.+)(?=\s*).*\1 to " HEART...
1
by: Allan Ebdrup | last post by:
I have a dynamic list of regular expressions, the expressions don't change very often but they can change. And I have a single string that I want to match the regular expressions against and find...
5
by: shawnmkramer | last post by:
Anyone every heard of the Regex.IsMatch and Regex.Match methods just hanging and eventually getting a message "Requested Service not found"? I have the following pattern: ^(?<OrgCity>(+)+),...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.