HTMLPurifier - Standard Compliant HTML Filtering

[Ambush Commander]

HTMLPurifier is a new PHP library that filters HTML so that not only is
XSS thwarted, but the resulting HTML is standards-compliant! It's
licensed under LGPL, and is currently undergoing beta testing (beta
meaning that validation routines for a few shorthand CSS properties and
deprecated HTML properties are missing, but everything else is there).

The main difference from HTMLPurifier is that while older packages like
kses and HTML_Safe attempt to blacklist XSS, HTMLPurifier employs a
whitelist approach, breaking down an HTML document and rigorously
testing everything, whether it be a color declaration or an external
URI.

Try it out first: http://hp.jpsband.org/live/docs/examples/demo.php
Then grab a copy here: http://hp.jpsband.org/

[Ambush Commander]

Ehh... sorry about the parse error. The demo is running off code in the
trunk (which is undergoing active development right now).

[John Dunlop]

Ambush Commander:

HTMLPurifier is a new PHP library that filters HTML so that not only is
XSS thwarted, but the resulting HTML is standards-compliant!

Do you mean standards compliant, valid or something else? If you mean
standards compliant - assuming that that includes HTML - you would have
to assign meanings to all the ambiguous clauses of the HTML4.01 spec
(strictly speaking, all of them). If you mean valid, you would have to
guess or somehow infer what any invalid markup was intended to mean
before you could sort it.

--
Jock

[Ambush Commander]

John Dunlop wrote:

Do you mean standards compliant, valid or something else? If you mean
standards compliant - assuming that that includes HTML - you would have
to assign meanings to all the ambiguous clauses of the HTML4.01 spec
(strictly speaking, all of them). If you mean valid, you would have to
guess or somehow infer what any invalid markup was intended to mean
before you could sort it.

In a way, both. I can't be completely standards compliant, because
technically that would mean I'd let XSS through. What I can do is,
while disallowing XSS, ensure that any output the filter gives won't
break a XHTML 1.0 Transitional page's validation at the W3C validator.
This is no easy task, especially since the spec doesn't get everything
right (for example SGML exclusions). Currently, the only thing that's
bothering the filter are control characters and non-SGML allowed
codepoints: anything else you throw at it will be turned into something
that will validate.

As in valid, people use deprecated elements and attributes like <font>
and <centerall the time. The filter converts these into their proper
representations (<span style=""and <div style="text-align:center;">)
So it can be quite smart about that sort of thing (it also does
automatic <ptag closings, etc). Kind of like Tidy, the only thing is
that Tidy doesn't guarantee validation. We do.

[John Dunlop]

Ambush Commander:

In a way, both. I can't be completely standards compliant, because
technically that would mean I'd let XSS through. What I can do is,
while disallowing XSS, ensure that any output the filter gives won't
break a XHTML 1.0 Transitional page's validation at the W3C validator.
This is no easy task, especially since the spec doesn't get everything
right (for example SGML exclusions). Currently, the only thing that's
bothering the filter are control characters and non-SGML allowed
codepoints: anything else you throw at it will be turned into something
that will validate.

I don't mean to sound rude, but what is this 'something'? How do you
know when you come across an error what was originally meant? Do you
flag the error and ask the user what they meant?

As in valid, people use deprecated elements and attributes like <font>
and <centerall the time. The filter converts these into their proper
representations (<span style=""and <div style="text-align:center;">)
So it can be quite smart about that sort of thing (it also does
automatic <ptag closings, etc). Kind of like Tidy, the only thing is
that Tidy doesn't guarantee validation. We do.

I don't believe there is any program today that can check conformance
to the HTML spec. Machines have no understanding of the prose of the
spec. Your program, from what I gather, checks validity and a
selection of other criteria that you have chosen: a linter with built
in validator.

--
Jock

[Ambush Commander]

John Dunlop wrote:

I don't mean to sound rude, but what is this 'something'? How do you
know when you come across an error what was originally meant? Do you
flag the error and ask the user what they meant?

No, we just mangle it and hope the user notices. :-P Error logging and
feedback is a feature I'd like to implement soon.

I don't believe there is any program today that can check conformance
to the HTML spec. Machines have no understanding of the prose of the
spec.

But the /programmer/ can. I manually went through the HTML and CSS
specs and hand-picked the elements, attributes and properties that
would be acceptable from an untrusted user in a rich text environment.
And then hand-coded their definitions.

Your program, from what I gather, checks validity and a
selection of other criteria that you have chosen: a linter with built
in validator.

Hmm... I don't see how that's much different from a filter. It won't
"fix" an excessive use and duplication of inline styles. It can't
figure out that a user is abusing a certain tag for a different
meaning. But it will make a document conform in the eyes of the W3C
validator, and it will block XSS attempts (by virtue of its whitelist
nature).

I feel like a salesperson trying to "sell" a product. Please feel free
to ask more questions.