url encoding string (Code Worth Recommending Project)

[Peter Michaux]

encodeURIComponent is commonly used to serialize forms for use with
XMLHttpRequest requests. For a perfect simulation of browser form
requests, the goal is to serialize form data in the <URL:
http://www.w3.org/TR/1999/REC-html401-19991224/interact/forms.html#h-17.13.4.1>
application/x-www-form-urlencoded</astandardized format.

The handling of whitespace by encodeURIComponent() is different from
the x-www-form-urlencoded standard. According to the x-www-form-
urlencoded standard, a space should be encoded as a "+" and a newline
should be encoded as a "%0D%0A".

The JavaScript encodeURIComponent function encodes a space as "%20".
On Windows operating system a new line is encoded as "%0D%0A" but on
Mac OS X a new line is encoded as just "%0A". I don't know about
Linux. Can someone check using a textarea in a form?

Some developers may need to have the application/x-www-form-urlencoded
format. We can wrap encodeURIComponent.

if (typeof encodeURIComponent != 'undefined' &&
String.prototype.replace) {

var urlEncode = function(s) {
return encodeURIComponent(s).replace(/%20/, '+').replace(/(.
{0,3})(%0A)/g,
function(m, p1, p2) {return p1+(p1=='%0D'?'':'%0D')+p2;});
};
})();

}

The feature detection for String.prototype.replace is not sufficient
because we really need to know if the second argument to replace can
be a function. I imagine we can test this by using try-catch and
actually using replace with a function as the second argument. Does
anyone know a way to check without using try-catch?
For developers with server-side toolkits capable of dealing with the
output of encodeURIComponent the regexp work is not necessary and can
use just this simple little function.

if (typeof encodeURIComponent != 'undefined') {

var urlEncode = function(s) {
return encodeURIComponent(s);
};

}
There could even be a version for forms without textareas so only the
spaces need replacement.

if (typeof encodeURIComponent != 'undefined' &&
String.prototype.replace) {

var urlEncode = function(s) {
return encodeURIComponent(s).replace(/%20/, '+');
};
})();

}

--------------

BTW, I set up svn/trac for the code and posted the getOptionValue code
that resulted from the previous thread on form serialization.

<URL: http://cljs.michaux.ca/trac/browser/trunk/src/getOptionValue>

Please email me if you would like wiki editing and ticket creation/
commenting permissions. Let me know if you have a username/password
preference.

Peter

[pr]

Peter Michaux wrote:

On Windows operating system a new line is encoded as "%0D%0A" but on
Mac OS X a new line is encoded as just "%0A". I don't know about
Linux. Can someone check using a textarea in a form?

Same as Windows: before%0D%0Aafter

Linux xxxx 2.6.17-12-generic #2 SMP Sun Sep 23 22:56:28 UTC 2007 i686
GNU/Linux

[Bart Van der Donck]

Peter Michaux wrote:

The JavaScript encodeURIComponent function encodes a space as "%20".
On Windows operating system a new line is encoded as "%0D%0A" but on
Mac OS X a new line is encoded as just "%0A". I don't know about
Linux. Can someone check using a textarea in a form?

Yes but %0A is only because Mac OSX is a UNIX derivate (which
historically always used %0A).

Mac until OS9 should return %0D.

http://en.wikipedia.org/wiki/Line_Feed

%0A: Multics, Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac
OS X, etc.), BeOS, Amiga, RISC OS, and others
%0D%0A: DEC RT-11 and most other early non-Unix, non-IBM OSes, CP/M,
MP/M, MS-DOS, OS/2, Microsoft Windows
%0D: Commodore machines, Apple II family and Mac OS up to version 9

--
Bart

[Peter Michaux]

On Nov 21, 1:44 am, Bart Van der Donck <[e-mail]:

Peter Michaux wrote:

The JavaScript encodeURIComponent function encodes a space as "%20".
On Windows operating system a new line is encoded as "%0D%0A" but on
Mac OS X a new line is encoded as just "%0A". I don't know about
Linux. Can someone check using a textarea in a form?

Yes but %0A is only because Mac OSX is a UNIX derivate (which
historically always used %0A).

Mac until OS9 should return %0D.

http://en.wikipedia.org/wiki/Line_Feed

%0A: Multics, Unix and Unix-like systems (GNU/Linux, AIX, Xenix, Mac
OS X, etc.), BeOS, Amiga, RISC OS, and others
%0D%0A: DEC RT-11 and most other early non-Unix, non-IBM OSes, CP/M,
MP/M, MS-DOS, OS/2, Microsoft Windows
%0D: Commodore machines, Apple II family and Mac OS up to version 9

Thanks pr and Bart. It seems the two options ('%0D' and '%0A') need to
be corrected. How about this?

var urlencode = (function() {

var f = function(s) {
return encodeURIComponent(s).replace(/%20/,
'+').replace(/(.{0,3})(%0A)/g,
function(m, p1, p2) {return p1+(p1=='%0D'?'':'%0D')+p2;}
).replace(/(%0D)(.{0,3})/g,
function(m, p1, p2) {return p1+(p2=='%0A'?'':'%0A')+p2;});
};

try {
if (f('\n \r') == '%0D%0A+%0D%0A') {
return f;
}
}
catch(e) {}

})();
Using try-catch is a nice way to avoid host object feature testing and
ensure that all the necessary parts of the regexp are working as
expected. This makes a full test of the new feature in a very small
space.

However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

Peter

[pr]

Peter Michaux wrote:

However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

If it's backwards-compatibility you mainly want to test for, then AFAIK
try...catch arrived in the language about the same time as the ability
to use a function in String.replace() (JS 1.3 & 1.4, IE 5 & 5.5). So if
the feature isn't present then a try...catch feature test is almost
equally likely to fall over.

Unless you expect somebody with a seven-year-old browser to show up or
you know of a recent implementation that can't successfully use
functions in String.replace(), it would seem reasonable not to test at
all. I couldn't think of a test for this that would work pre-Netscape
6.0 without browser sniffing.

[Peter Michaux]

On Nov 22, 5:28 am, pr <[e-mail]:

Peter Michaux wrote:

However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

If it's backwards-compatibility you mainly want to test for, then AFAIK
try...catch arrived in the language about the same time as the ability
to use a function in String.replace() (JS 1.3 & 1.4, IE 5 & 5.5). So if
the feature isn't present then a try...catch feature test is almost
equally likely to fall over.

Unless you expect somebody with a seven-year-old browser to show up or
you know of a recent implementation that can't successfully use
functions in String.replace(), it would seem reasonable not to test at
all. I couldn't think of a test for this that would work pre-Netscape
6.0 without browser sniffing.

Randy Webb frequently mentions his dislike for try-catch and says his
cell phone doesn't support it. I really don't have a problem with try-
catch other than I don't like using features when they aren't
necessary.

It seems that if a browser doesn't support a function second argument
then this would be sufficient.

var urlencode = (function() {

var f = function(s) {
return encodeURIComponent(s).replace(/%20/,
'+').replace(/(.{0,3})(%0A)/g,
function(m, p1, p2) {return p1+(p1=='%0D'?'':'%0D')+p2;}
).replace(/(%0D)(.{0,3})/g,
function(m, p1, p2) {return p1+(p2=='%0A'?'':'%0A')+p2;});
};

if (f('\n \r') == '%0D%0A+%0D%0A') {
return f;
}

})();

The reason this is ok is because the if conditional will be false (not
throw an error). JavaScript will automatically type convert the
function second argument of replace to a string.

"asdf".replace(/s/, function(){});

outputs something like this

afunction(){}df

The Function.prototype.toString function is standard but it's behavior
is implementation dependent.

So I think try-catch may not be necessary after all.

Peter

[Randy Webb]

Peter Michaux said the following on 11/22/2007 10:06 AM:

On Nov 22, 5:28 am, pr <[e-mail]:

>Peter Michaux wrote:

>>However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

If it's backwards-compatibility you mainly want to test for, then AFAIK
try...catch arrived in the language about the same time as the ability
to use a function in String.replace() (JS 1.3 & 1.4, IE 5 & 5.5). So if
the feature isn't present then a try...catch feature test is almost
equally likely to fall over.

Unless you expect somebody with a seven-year-old browser to show up or
you know of a recent implementation that can't successfully use
functions in String.replace(), it would seem reasonable not to test at
all. I couldn't think of a test for this that would work pre-Netscape
6.0 without browser sniffing.

Randy Webb frequently mentions his dislike for try-catch and says his
cell phone doesn't support it. I really don't have a problem with try-
catch other than I don't like using features when they aren't
necessary.

It auto-updated itself about a week ago and according to Motorola any
cell phone that has that browser was also auto-updated. I don't think I
have to say I am glad it happened.

It supports about 90% of what IE7 supports, including try/catch.

--
Randy
Chance Favors The Prepared Mind
comp.lang.javascript FAQ - http://jibbering.com/faq/index.html
Javascript Best Practices - http://www.JavascriptToolbox.com/bestpractices/

[Thomas 'PointedEars' Lahn]

pr wrote:

Peter Michaux wrote:

>However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

If it's backwards-compatibility you mainly want to test for, then AFAIK
try...catch arrived in the language about the same time as the ability
to use a function in String.replace()

No, it did not. How did you get that strange idea?

(JS 1.3 & 1.4, IE 5 & 5.5).

Your figures are way off.

http://PointedEars.de/es-matrix#try
PointedEars
--
var bugRiddenCrashPronePieceOfJunk = (
navigator.userAgent.indexOf('MSIE 5') != -1
&& navigator.userAgent.indexOf('Mac') != -1
) // Plone, register_function.js:16

[Peter Michaux]

On Nov 23, 10:08 am, Thomas 'PointedEars' Lahn <[e-mail]>
wrote:

pr wrote:

Peter Michaux wrote:

However, try-catch doesn't appeal to some programmers. Any other
option to test that replace can take a function for the second
argument?

If it's backwards-compatibility you mainly want to test for, then AFAIK
try...catch arrived in the language about the same time as the ability
to use a function in String.replace()

No, it did not. How did you get that strange idea?

(JS 1.3 & 1.4, IE 5 & 5.5).

Your figures are way off.

http://PointedEars.de/es-matrix#try

Nice page but I don't see anything about when String.prototype.replace
could take a function as the second argument.

Peter

[Thomas 'PointedEars' Lahn]

Thomas 'PointedEars' Lahn wrote:

Peter Michaux wrote:

>On Nov 23, 10:08 am, Thomas 'PointedEars' Lahn <[e-mail]>

>>http://PointedEars.de/es-matrix#try

[...] I don't see anything about when String.prototype.replace
could take a function as the second argument.

Implicitly, it does. That feature could not have been available before
function expressions were introduced. (I might emphasize that by stating
it in a row below explicitly, but then I would have to do it for several
methods.)

I have spotted a bug in the Matrix ;-)

String.prototype.replace() was only described with a string as second
argument, which probably has contributed not only to your confusion.
Sorry about that.

I have done some tests now and updated the ES Matrix accordingly. I have
also fixed an error (of omission) at MDC: the method was not part of
ECMAScript specifications before Edition *3*.

http://developer.mozilla.org/en/docs...eplace#Summary
PointedEars
--
Prototype.js was written by people who don't know javascript for people
who don't know javascript. People who don't know javascript are not
the best source of advice on designing systems that use javascript.
-- Richard Cornford, cljs, <f8*******************@news.demon.co.uk>

[Richard Cornford]

Thomas 'PointedEars' Lahn wrote:

Peter Michaux wrote:

<snip>

>but I don't see anything about when String.prototype.replace
could take a function as the second argument.

Implicitly, it does. That feature could not have been
available before function expressions were introduced.

<snip>

The second argument to - String.prototype.replace - does not have to be
a function expression. It can be (and very often should be) a reference
to a function via a named property of a Variable object (an Identifier
in the actual code). You can do that as soon as you have first class
functions (as function objects) that can be passed by reference, which
is pretty much from day one.

Richard.

[Richard Cornford]

Peter Michaux wrote:
<snip>

The reason this is ok is because the if conditional will be
false (not throw an error). JavaScript will automatically
type convert the function second argument of replace to a
string.

"asdf".replace(/s/, function(){});

outputs something like this

afunction(){}df

The Function.prototype.toString function is standard but
it's behavior is implementation dependent.

So I think try-catch may not be necessary after all.

Given the number of times I have seen you referring people to my page in
the FAQ notes about feature detecting it is disappointing that you have
forgotten that the example of pure javascript feature detection that I
used on that page was a test for - String.prototype.replace - accepting
a function reference as its second argument or not (and it did not use
try-catch).

Richard.

[Thomas 'PointedEars' Lahn]

Richard Cornford wrote:

Thomas 'PointedEars' Lahn wrote:

>Peter Michaux wrote:

<snip>

>>but I don't see anything about when String.prototype.replace
could take a function as the second argument.

Implicitly, it does. That feature could not have been
available before function expressions were introduced.

<snip>

The second argument to - String.prototype.replace - does not have to be
a function expression. It can be (and very often should be) a reference
to a function via a named property of a Variable object (an Identifier
in the actual code).

"Very often should be" -- why? That would pollute the namespace and, given
the current support of runtime environments, it would pollute it needlessly.
I would have concurred if you had said "there are occasions where it is
advisable" but not with this.

You can do that as soon as you have first class functions (as function
objects) that can be passed by reference, which is pretty much from day one.

Nevertheless, String.prototype.replace() was not introduced before
JavaScript 1.2, JScript 1.0, and ECMAScript 3, and function expressions
were introduced with JavaScript 1.2, JScript 3.0, and ECMAScript 3. So
yes, it looks like there is a correlation.
PointedEars
--
var bugRiddenCrashPronePieceOfJunk = (
navigator.userAgent.indexOf('MSIE 5') != -1
&& navigator.userAgent.indexOf('Mac') != -1
) // Plone, register_function.js:16

[Peter Michaux]

On Nov 24, 8:50 am, "Richard Cornford" <[e-mail]>
wrote:

Peter Michaux wrote:

<snip>

The reason this is ok is because the if conditional will be
false (not throw an error). JavaScript will automatically
type convert the function second argument of replace to a
string.

"asdf".replace(/s/, function(){});

outputs something like this

afunction(){}df

The Function.prototype.toString function is standard but
it's behavior is implementation dependent.

So I think try-catch may not be necessary after all.

Given the number of times I have seen you referring people to my page in
the FAQ notes about feature detecting it is disappointing that you have
forgotten that the example of pure javascript feature detection that I
used on that page was a test for - String.prototype.replace - accepting
a function reference as its second argument or not (and it did not use
try-catch).

At least I had the joy of rediscovery :-)

Peter