Extract any URL from any string?

[deko]

geturl.php

Too much code to paste here, but have a look at http://www.liarsscourge.com/

So far, I have not found a string that can break this...

Any built-in functions or suggestions for improvement?

Thanks in advance.

[Daz]

On Feb 10, 8:41 pm, "deko" <[e-mail] mwrote:

geturl.php

Too much code to paste here, but have a look athttp://www.liarsscourg e.com/

So far, I have not found a string that can break this...

Any built-in functions or suggestions for improvement?

Thanks in advance.

I don't want to sound negative here, but what exactly is the point?
The reason I ask, is because I see no reason why you can't extract it
with a single regex expression, and then you could use another one to
validate it. It's simple enough to validate, the question is how valid
do you want it to be. Should you need to specify each TLD, or do you
just need to match a pattern. In either case, 2 or 3 regex expressions
max should be able to do what you are after. With a little extra
crafting, you should be able to extract multiple URLs in one go.

[Janwillem Borleffs]

deko wrote:

geturl.php

Too much code to paste here, but have a look at
http://www.liarsscourge.com/
So far, I have not found a string that can break this...

Any built-in functions or suggestions for improvement?

1. Increase the error_reporting level and you will find some sloppy notices
2. Have a look at parse_url(), which might be useful
3. Use preg_* functions instead of POSIX ereg* function (performance)
4. Strings like the following cause infinite loops:

getURL('fofo http://discovery.co.uk/../foo');

Probable fix:

= Replace:

if (!eregi("^(com| net|org...)$", $urlString_a[$i])) {
...
}

= With:

if (preg_match("!^ (com|net|org... )[^$]!", $urlString_a[$i], $m)) {
$urlString_a[$i] = $m[1];
}
JW

[deko]

1. Increase the error_reporting level and you will find some sloppy notices

2. Have a look at parse_url(), which might be useful
3. Use preg_* functions instead of POSIX ereg* function (performance)
4. Strings like the following cause infinite loops:

getURL('fofo http://discovery.co.uk/../foo');

Probable fix:

= Replace:

if (!eregi("^(com| net|org...)$", $urlString_a[$i])) {
...
}

= With:

if (preg_match("!^ (com|net|org... )[^$]!", $urlString_a[$i], $m)) {
$urlString_a[$i] = $m[1];
}

Outstanding. Thanks for the constructive feedback.

[deko]

geturl 2.0...

1. Increase the error_reporting level and you will find some sloppy notices

so I have a few undefined variables... I thought this did not matter with PHP
(script vs. compiled code)

2. Have a look at parse_url(), which might be useful

well, if urlString = http://netforex.subdom ain.net.foo'>, parse_url() returns:
netforex.subdom ain.net.foo'- which is not much help. Nevertheless, I'm using
it on line 79 for validation - but I think it would be better validation if I
could do this:
if ($urlLink = parse_url($urlL ink))
{
return $urlLink['host'];
}
??

3. Use preg_* functions instead of POSIX ereg* function (performance)

I was able to replace an eregi with preg_match on lines 18 and 29, but unsure of
the syntax for line 44 ... suggestions?

4. Strings like the following cause infinite loops:

getURL('fofo http://discovery.co.uk/../foo');

fixed! see line 38.

- - - - -

I'm wondering how much of a hack that for loop is (lines 26 - 53). Seems to
work okay...

[deko]

http://www.liarsscourge.com <<== the script is here
"deko" <de**@nospam.co mwrote in message
news:5K******** *************** *******@comcast .com...

geturl 2.0...

>1. Increase the error_reporting level and you will find some sloppy notices

so I have a few undefined variables... I thought this did not matter with PHP
(script vs. compiled code)

>2. Have a look at parse_url(), which might be useful

well, if urlString = http://netforex.subdom ain.net.foo'>, parse_url()
returns: netforex.subdom ain.net.foo'- which is not much help. Nevertheless,
I'm using it on line 79 for validation - but I think it would be better
validation if I could do this:
if ($urlLink = parse_url($urlL ink))
{
return $urlLink['host'];
}
??

>3. Use preg_* functions instead of POSIX ereg* function (performance)

I was able to replace an eregi with preg_match on lines 18 and 29, but unsure
of the syntax for line 44 ... suggestions?

>4. Strings like the following cause infinite loops:

getURL('fofo http://discovery.co.uk/../foo');

fixed! see line 38.

- - - - -

I'm wondering how much of a hack that for loop is (lines 26 - 53). Seems to
work okay...

[Janwillem Borleffs]

deko wrote:

so I have a few undefined variables... I thought this did not matter
with PHP (script vs. compiled code)

It doesn't, but it's just good practice to do so (and increasing the
error_reporting level saves you a lot of time debugging when making typos in
variable names).

well, if urlString = http://netforex.subdom ain.net.foo'>,
parse_url() returns: netforex.subdom ain.net.foo'- which is not much

The parse_url function can indeed easily be fooled, so you will have to add
some additional validation. But it's quite useful for what you are doing
(and pretty fast).

I was able to replace an eregi with preg_match on lines 18 and 29,
but unsure of the syntax for line 44 ... suggestions?

You mean the following line?

if (eregi("^(com|n et|org...)$", $urlString_a[$i]))

You can simply replace this with:

if (preg_match("!^ (com|net|org... )$!i", $urlString_a[$i]))

JW

[deko]

>so I have a few undefined variables... I thought this did not matter

>with PHP (script vs. compiled code)

It doesn't, but it's just good practice to do so (and increasing the
error_reporting level saves you a lot of time debugging when making typos in
variable names).

Understood. Adjusting error_reporting is helpful for debugging and
optimization.

>well, if urlString = http://netforex.subdom ain.net.foo'>,
parse_url() returns: netforex.subdom ain.net.foo'- which is not much

The parse_url function can indeed easily be fooled, so you will have to add
some additional validation. But it's quite useful for what you are doing (and
pretty fast).

I'm not sure where parse_url is useful, precisely because it is so easily
fooled. The
strings I'm working with are entirely unpredictable. That's why I explode the
candidate URL into domains and inspect each one in a for loop.

>I was able to replace an eregi with preg_match on lines 18 and 29,
but unsure of the syntax for line 44 ... suggestions?

You mean the following line?

>if (eregi("^(com|n et|org...)$", $urlString_a[$i]))

You can simply replace this with:

>if (preg_match("!^ (com|net|org... )$!i", $urlString_a[$i]))

Thanks, but I'm still baffled by that syntax - why the second negation '!' after
the first '$' ? I understand that '$' means end of line, and 'i' means case
insensitive... is '$!' saying "not ending with"?

I still want to replace the eregi code (lines 44, 51, 55, 85) with preg_match,
but the preg_match syntax seems counter intuitive... just have not figured it
out yet. What's up with delimiters?

http://www.liarsscourge.com <<== latest code here

Thanks again for the help!

[deko]

http://www.liarsscourge.com/ <<== this is better

known bug: if an email address appears in the test string before a valid URL,
the script will not find the URL

[Curtis]

deko wrote:

http://www.liarsscourge.com/ <<== this is better

known bug: if an email address appears in the test string before a valid
URL, the script will not find the URL

Hard coding TLDs is generally not useful, as you never know when
unexpected ones may be put in use. Plus, you do not allow variance for
different schemes other than http(s).

You do not support valid URLs that have the authority:

<http://bob1234z_.sss:li*****@example.com/foo/bar
baz/index.bak.php?q =my&q2=query#fr ag2>

This is indeed a valid URL, but your algorithm fails. It's far more
useful to use a single regex, anticipate any scheme (look at wikipedia
or search some RFCs for valid URI format), and any TLD.

parse_url is not meant to be used for validation, as stated in the PHP
docs themselves.

This is an example implementing the regex I made, recently:

<?php
$re = '%
( [\w.+-]+ : (?://)? ) # scheme name

( [^/]+ ) # authority, domain
( / [^?]+ )? # path, if exists

# query and fragment, which may or may not exist
(?:
\\? # query initializer
( [^#]+ ) # grab query
(?: \\# ([\w-]+) )? # fragment, if exists
)?
%x';

$s = 'Welcome.to{"ht tp://user:pa**@examp le.com/foo
bar".-/index.bak.php?q =query&r=arrr#f rag2_2borky borked!';
if (preg_match($re , $s, $m)) {
echo '<p>Original: <code>' . $s . '</code></p>';
echo '<p>Extrapolati on: <a href="'
. htmlentities($m[0], ENT_QUOTES) . '">' . ($m[1].$m[2])
. '</a(full URI in link, see status bar).</p>';
}
else {
echo 'Not a valid URI.';
}
?>

Curtis