473,232 Members | 1,561 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,232 software developers and data experts.

Regular Expression Matches

I'm using regular expressions to extract some data and some links from some
web pages. I download the page and then I want to get a list of certain
links.

For building regular expressions, I use an app call The Regulator, which
makes it pretty easy to build and test regular expressions.

As a warning, I'm real weak with regular expressions. Let's say my regular
expression is:

(href=)(?<caturl>.*)(class=title>\[&nbsp;&nbsp;&nbsp;)

Now, using The Regulator and giving it the source for a particular web page,
I get 8 matches.

According to the regulator, the options it's using are:

Multiline, ignore case, ignore whitespace

In my own code, I'm doing:

Regex indexRegex = new Regex(categoryListRegex,
RegexOptions.Multiline |
RegexOptions.IgnorePatternWhitespace |
RegexOptions.IgnoreCase);
MatchCollection indexMatches = indexRegex.Matches(pageText);

This only returns one match in indexMatches with the same page that I'm
giving The Regulator. It seems that no matter what combination of regex
options I use, I'm only getting one match.

Why is that? How do I get all 8 matches?

Thanks.

pete
Feb 21 '06 #1
9 3314
Hi Pete,

You need to escape the '<' and '>' characters in your Regurlar Expression.
These are used in some flavors of Regular Expression language to indicate a
named group. If the first (<caturl>) is a group name, name both groups or
neither.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:1e******************************@giganews.com ...
I'm using regular expressions to extract some data and some links from
some web pages. I download the page and then I want to get a list of
certain links.

For building regular expressions, I use an app call The Regulator, which
makes it pretty easy to build and test regular expressions.

As a warning, I'm real weak with regular expressions. Let's say my regular
expression is:

(href=)(?<caturl>.*)(class=title>\[&nbsp;&nbsp;&nbsp;)

Now, using The Regulator and giving it the source for a particular web
page, I get 8 matches.

According to the regulator, the options it's using are:

Multiline, ignore case, ignore whitespace

In my own code, I'm doing:

Regex indexRegex = new Regex(categoryListRegex,
RegexOptions.Multiline |
RegexOptions.IgnorePatternWhitespace |
RegexOptions.IgnoreCase);
MatchCollection indexMatches = indexRegex.Matches(pageText);

This only returns one match in indexMatches with the same page that I'm
giving The Regulator. It seems that no matter what combination of regex
options I use, I'm only getting one match.

Why is that? How do I get all 8 matches?

Thanks.

pete

Feb 21 '06 #2
Well, there are 3 groups. <caturl> is a group name. The other 2 are unnamed.
Why do they need to be named if <caturl> is named? I'm not interested in the
other groups. I'm simply using them as "delimiters" for lack of a better
word.

I've modified the expression to look like this:

(href=)(?<caturl>.*)(class=title.*\[&nbsp;&nbsp;&nbsp;)

This gives the exact same results.

The escaping stuff gets a little confusing because the regular expressions
are actually stored in an XML file, so they get escaped for that.

In the XML file that looks like:

(href=)(?&lt;caturl&gt;.*)(class=title.*\[&amp;nbsp;&amp;nbsp;&amp;nbsp;)

This still isn't returning multiple results. Just the last match. I don't
think the < was the problem.

Pete

"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:%2****************@TK2MSFTNGP11.phx.gbl...
Hi Pete,

You need to escape the '<' and '>' characters in your Regurlar Expression.
These are used in some flavors of Regular Expression language to indicate
a named group. If the first (<caturl>) is a group name, name both groups
or neither.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:1e******************************@giganews.com ...
I'm using regular expressions to extract some data and some links from
some web pages. I download the page and then I want to get a list of
certain links.

For building regular expressions, I use an app call The Regulator, which
makes it pretty easy to build and test regular expressions.

As a warning, I'm real weak with regular expressions. Let's say my
regular expression is:

(href=)(?<caturl>.*)(class=title>\[&nbsp;&nbsp;&nbsp;)

Now, using The Regulator and giving it the source for a particular web
page, I get 8 matches.

According to the regulator, the options it's using are:

Multiline, ignore case, ignore whitespace

In my own code, I'm doing:

Regex indexRegex = new Regex(categoryListRegex,
RegexOptions.Multiline |
RegexOptions.IgnorePatternWhitespace |
RegexOptions.IgnoreCase);
MatchCollection indexMatches = indexRegex.Matches(pageText);

This only returns one match in indexMatches with the same page that I'm
giving The Regulator. It seems that no matter what combination of regex
options I use, I'm only getting one match.

Why is that? How do I get all 8 matches?

Thanks.

pete


Feb 21 '06 #3
I found the solution to the problem, so now if someone could explain why,
I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete
Feb 22 '06 #4
> The solution is to replace carriage returns with line feeds.

I didn't see any carriage returns or line feeds in your regular expression.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:pK********************@giganews.com...
I found the solution to the problem, so now if someone could explain why,
I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete

Feb 22 '06 #5
The Carriage Returns and Linefeeds are in the web page I'm scanning, not the
regex.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:u4*************@tk2msftngp13.phx.gbl...
The solution is to replace carriage returns with line feeds.


I didn't see any carriage returns or line feeds in your regular
expression.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:pK********************@giganews.com...
I found the solution to the problem, so now if someone could explain why,
I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete


Feb 22 '06 #6
Well, Pete, I'm not sure, but I do know that Microsoft and Unix text
documents have a distinct difference in terms of line feeds. The Microsoft
text document uses CR/LF (\r\n), while the Unix model uses only LF(\n). Does
that tell you anything?

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:Re******************************@giganews.com ...
The Carriage Returns and Linefeeds are in the web page I'm scanning, not
the regex.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:u4*************@tk2msftngp13.phx.gbl...
The solution is to replace carriage returns with line feeds.


I didn't see any carriage returns or line feeds in your regular
expression.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:pK********************@giganews.com...
I found the solution to the problem, so now if someone could explain why,
I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete



Feb 22 '06 #7
Kevin,

Thanks for the help. The Unix/Windows difference on crlf vs. lf I'm aware
of, but that's not really the issue.

The Regex engine in .NET, I would assume, is more geared towards windows
documents if anything, but maybe not.

There are Regex options for singleline and multiline to help determine how
the parser handles the data, but it's unclear to me how to get all matches
from my document without replacing the carriage returns. Nothing I've tried
has worked. I suspect there are changes I can make to the regular
expressions themselves to handle it, but I don't know how to do that.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:Os****************@TK2MSFTNGP12.phx.gbl...
Well, Pete, I'm not sure, but I do know that Microsoft and Unix text
documents have a distinct difference in terms of line feeds. The Microsoft
text document uses CR/LF (\r\n), while the Unix model uses only LF(\n).
Does that tell you anything?

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:Re******************************@giganews.com ...
The Carriage Returns and Linefeeds are in the web page I'm scanning, not
the regex.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:u4*************@tk2msftngp13.phx.gbl...
The solution is to replace carriage returns with line feeds.

I didn't see any carriage returns or line feeds in your regular
expression.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:pK********************@giganews.com...
I found the solution to the problem, so now if someone could explain
why, I'd appreciate that:

The solution is to replace carriage returns with line feeds.

If I do that, I get all 8 results instead of the 1 result I was
getting.

Now, can anyone tell me WHY? And aren't there other regex options for
dealing with that without having to change the source text?

Thanks.

Pete



Feb 23 '06 #8

Pete Davis wrote:
I'm using regular expressions to extract some data and some links from some
web pages. I download the page and then I want to get a list of certain
links.


Completely not answering your actual question, but maybe this will save
you a lot of bother: the free HtmlAgilityPack will convert an (even
malformed) HTML document into a nice XML tree which makes getting
content out much much easier.

--
Larry Lard
Replies to group please

Feb 23 '06 #9
Hi Pete,

Here are some Regular Expressions that may help with this sort of thing. I
tend to use these rather than the options in the Regex engine:

(?i) Turn of case-sensitivity for the remainder
of the regular expression.
(?s) Turn on "dot matches now line" for the
remainder of the regular expression.
(?m) Caret and dollar sign match after and before
new lines for the remainder of the regular expression.
(?i-sm) Tunrs on the "i" and "m" options for the remainder
of the regular expression, and turn off "s"
(?i-sm:regex) Turns on the "i" and "m" options
for the regular expression inside the parentheses,
and turn off "s"

Here is the Microsoft .Net Framework Regular Expressions reference:

http://msdn.microsoft.com/library/de...geElements.asp

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:aa********************@giganews.com...
Kevin,

Thanks for the help. The Unix/Windows difference on crlf vs. lf I'm aware
of, but that's not really the issue.

The Regex engine in .NET, I would assume, is more geared towards windows
documents if anything, but maybe not.

There are Regex options for singleline and multiline to help determine how
the parser handles the data, but it's unclear to me how to get all matches
from my document without replacing the carriage returns. Nothing I've
tried has worked. I suspect there are changes I can make to the regular
expressions themselves to handle it, but I don't know how to do that.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:Os****************@TK2MSFTNGP12.phx.gbl...
Well, Pete, I'm not sure, but I do know that Microsoft and Unix text
documents have a distinct difference in terms of line feeds. The
Microsoft text document uses CR/LF (\r\n), while the Unix model uses only
LF(\n). Does that tell you anything?

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:Re******************************@giganews.com ...
The Carriage Returns and Linefeeds are in the web page I'm scanning, not
the regex.

Pete
"Kevin Spencer" <ke***@DIESPAMMERSDIEtakempis.com> wrote in message
news:u4*************@tk2msftngp13.phx.gbl...
> The solution is to replace carriage returns with line feeds.

I didn't see any carriage returns or line feeds in your regular
expression.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
To a tea you esteem
a hurting back as a wallet.
"Pete Davis" <pdavis68@[nospam]hotmail.com> wrote in message
news:pK********************@giganews.com...
>I found the solution to the problem, so now if someone could explain
>why, I'd appreciate that:
>
> The solution is to replace carriage returns with line feeds.
>
> If I do that, I get all 8 results instead of the 1 result I was
> getting.
>
> Now, can anyone tell me WHY? And aren't there other regex options for
> dealing with that without having to change the source text?
>
> Thanks.
>
> Pete
>



Feb 23 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Kenneth McDonald | last post by:
I'm working on the 0.8 release of my 'rex' module, and would appreciate feedback, suggestions, and criticism as I work towards finalizing the API and feature sets. rex is a module intended to make...
4
by: Neri | last post by:
Some document processing program I write has to deal with documents that have headers and footers that are unnecessary for the main processing part. Therefore, I'm using a regular expression to go...
11
by: Dimitris Georgakopuolos | last post by:
Hello, I have a text file that I load up to a string. The text includes certain expression like {firstName} or {userName} that I want to match and then replace with a new expression. However,...
5
by: John | last post by:
I am new in Regular Expression. Could someone please help me in following expression? 1. the string cannot be empty 2. the string can only contains AlphaNumeric characters. No space or any...
5
by: Cylix | last post by:
I am going to write a function that the search engine done. in search engine, we may using double quotation to specify a pharse like "I love you", How can I using regular expression to sperate...
25
by: Mike | last post by:
I have a regular expression (^(.+)(?=\s*).*\1 ) that results in matches. I would like to get what the actual regular expression is. In other words, when I apply ^(.+)(?=\s*).*\1 to " HEART...
1
by: Allan Ebdrup | last post by:
I have a dynamic list of regular expressions, the expressions don't change very often but they can change. And I have a single string that I want to match the regular expressions against and find...
1
by: Shawn B. | last post by:
Greetings, I'm using a custom WebBrowser control: http://www.codeproject.com/KB/miscctrl/csEXWB.aspx When I get the DocumentSource of a web page I browsed, and run a regular expression...
14
by: Andy B | last post by:
I need to create a regular expression that will match a 5 digit number, a space and then anything up to but not including the next closing html tag. Here is an example: <startTag>55555 any...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.