Handling duplicate form submission

[nick4soup]

I have read the CGI FAQ 'How can I avoid users hitting "submit" twice'
(on
http://www.htmlhelp.org/faq/cgifaq.3.html#19 )
which essentially says you have to detect it at the server, using a
hidden form field.

That's fair enough.

My server is therefore processing a response on the first request, so
what kind of HTML response should I send out on the second, duplicate,
request?

Would this be easier if the response to the first request was a "Your
request is being processed ... [link to the response page]" with the
actual processing going on in the background?

[Using Javascript to prevent duplicate submissions is also noted, but
can't really be trusted]

Nick Bishop
-----
Email replies ignored.
-----
I didn't want a job vacuuming floors, but I got suckered into it
-oOo-

[Martin!]

ni*******@yahoo.com.au wrote:

I have read the CGI FAQ 'How can I avoid users hitting "submit" twice'
(on
http://www.htmlhelp.org/faq/cgifaq.3.html#19 )
which essentially says you have to detect it at the server, using a
hidden form field.

That's fair enough.

My server is therefore processing a response on the first request, so
what kind of HTML response should I send out on the second, duplicate,
request?

Would this be easier if the response to the first request was a "Your
request is being processed ... [link to the response page]" with the
actual processing going on in the background?

[Using Javascript to prevent duplicate submissions is also noted, but
can't really be trusted]

Nick Bishop
-----
Email replies ignored.
-----
I didn't want a job vacuuming floors, but I got suckered into it
-oOo-

you could disable the submit button when pressed once, then wait for the
server to send a reply.

[Alan J. Flavell]

On Mon, 28 Feb 2005, Martin! wrote (quoting what appeared to be the
whole posting - including sig - always a worrying sign, and often
suggesting that the respondent didn't actually bother to read properly
before responding):

ni*******@yahoo.com.au wrote:

I have read the CGI FAQ 'How can I avoid users hitting "submit" twice'
(on
http://www.htmlhelp.org/faq/cgifaq.3.html#19 )
which essentially says you have to detect it at the server, using a
hidden form field.

That's fair enough.
Indeed.
My server is therefore processing a response on the first request,
so what kind of HTML response should I send out on the second,
duplicate, request?

I think you've got the wrong mental picture here. HTTP itself is
stateless: each submission to the server, and its associated response,
is processed as a separate transaction, without any relationship to
any other transaction that happens to be going on in the same
timeframe. It's *your* task on the server side to tie these
transactions together, if/when there's any need to do so (and you're
implying that there is).

How you handle this in detail depends a lot on your system design, and
that in turn depends on how long you expect to take with processing a
transaction. If you can do a single transaction in the twinking of an
eye, then it's acceptable to put a lock on your database (for example)
and not handle any other transactions till it's complete. Then you
return the good response to the first submission, and start processing
the second. At this point you become aware that it's a duplicate, so
you reject it.

On the other hand, if the transaction is expected to take a while, you
may need some more-sophisticated strategy, so that you can start
looking at other transactions before you've committed the earlier
ones. The cited FAQ doesn't go into too much detail here.
you could disable the submit button when pressed once,

You could *try*, but it won't work reliably. Did you even bother to
look at the FAQ's answer?

[nick4soup]

Alan J. Flavell wrote:

My server is therefore processing a response on the first request, so what kind of HTML response should I send out on the second,
duplicate, request?
I think you've got the wrong mental picture here. HTTP itself is
stateless: each submission to the server, and its associated response, is processed as a separate transaction, without any relationship to
any other transaction that happens to be going on in the same
timeframe.
I think it's time for me to do some experiments. Set up a slow CGI
script that sends out two different responses, and see what happens
when I click twice.

It's *your* task on the server side to tie these
transactions together, if/when there's any need to do so (and you're
implying that there is).

With a database update, there is an absolute need. You don't want the
nonsense of being charged twice (for two searches, for example), or
adding something twice to a database.
you could disable the submit button when pressed once,

You could *try*, but it won't work reliably. Did you even bother to
look at the FAQ's answer?

I did, "Martin" probably didn't. It's a nice solution, but you've got
to cover the case for those without javascript (and anyway, I'm not
using Javascript at all in the first cut).

Nick.

[phil_gg04]

> How can I avoid users hitting "submit" twice

detect it at the server
what kind of HTML response should I send out on the second, duplicate, request?

This is an important issue. Lots of people will double-click on
buttons because that's what they have to do to some other elements in
their PC's user interface, and they forget which is which. Others get
impatient and try again if your server is being slow.

Ideally the system design will be such that the HTTP POSTs are
idempotent, i.e. the repeats are harmless. This can often be achieved
with a bit of thought, and avoids the issue entirely. Nick, if you
think you can't do this in your application, tell us some more about it
and maybe someone will suggest something.

If you can't do that I'd certainly have some javascript as that can
present a very user-friendly response to the majority of users. I have
gone for the idea of replacing the text of the submit button with the
word "Working..." after the first submit. I have even considered
making it blink. Replacing the entire page with some sort of
"Loading..." message is another option.

For duplicates that get past this you need a server-side mechanism to
detect them. This can be hard; for example, worry about things that
look like duplicates but are actually because the user has pressed BACK
and sent another genune request, and users who have disabled cookies.
If you do detect a duplicate, I think that the server should probably
just return a simple error message: "Error: duplicate request detected.
Did you press "submit" twice? Your first request only has been
processed.".

--Phil.

[Alan J. Flavell]

On Tue, 8 Mar 2005 ni*******@yahoo.com.au wrote:

Alan J. Flavell wrote:

I think you've got the wrong mental picture here. HTTP itself is
stateless: each submission to the server, and its associated response,

is processed as a separate transaction, without any relationship to
any other transaction that happens to be going on in the same
timeframe.

I think it's time for me to do some experiments.

See, the handling server-side of form submissions is rather more
appropriate to the group comp.infosystems.www.authoring.cgi (beware
its automoderation bot!) than to comp.infosystems.www.authoring.html
itself. That's true in practice (IMHO) whatever your server-side
technology happens to be, whether it's CGI as such, or whether it's
some other server-side technique such as mod_perl, PHP, ASP or
whatever turns you on, since the underlying principles are pretty much
the same.

But there are FAQs which address this specific point of duplicate
submissions.

http://www.htmlhelp.com/faq/html/forms.html#no-resubmit
http://www.cs.tut.fi/~jkorpela/forms/methods.html
http://www.htmlhelp.com/faq/cgifaq.3.html#19

It's *your* task on the server side to tie these
transactions together, if/when there's any need to do so (and you're
implying that there is).

With a database update, there is an absolute need.

Sure - don't make any mistake about this, I wasn't for a moment
suggesting that you didn't need it - quite the contrary, I was
suggesting that it's so important that you can't afford to rely on a
technique (i.e javascript) that might not work. The properties of
HTTP are stil what they are (i.e inherently stateless), and our job is
to deal with the consequences of that.

First and foremost, this is a non-idempotent request, so you should be
making it with a POST transaction. Client agents are *supposed* to
make some kind of provision for warning their users if they try to
repeat a POST transaction. Nevertheless, it would be good if you
could supply some protection of your own, such as a unique cookie in a
hidden field of the form. (Then it would be necessary for the users
to reload the form if they had a need to issue an additional request.
As a courtesy, don't forget to mention that in the instructions...)
I did, "Martin" probably didn't.
indeed
It's a nice solution,
It's a nice auxiliary convenience, but I couldn't agree that it's in
any real sense a "solution".
but you've got to cover the case for those without javascript (and
anyway, I'm not using Javascript at all in the first cut).

Just so. And in this case, as it happens, you won't actually need js
(which doesn't prevent you from adding it as an extra, if you want to,
but the key point is that you wouldn't be relying on it).

h t h

[nick4soup]

ph*******@treefic.com wrote:

Ideally the system design will be such that the HTTP POSTs are
idempotent, i.e. the repeats are harmless. This can often be achieved with a bit of thought, and avoids the issue entirely. Nick, if you
think you can't do this in your application, tell us some more about it and maybe someone will suggest something.

It looks as though a lot of people get it wrong.

Symptoms include being charged twice, or being double-subscribed, or a
horrible PRIMARY_KEY_VIOLATION error message that makes no sense to a
non-techo user (this comes because the logic tried to add it to the
database twice).

This suggests that Phil's (?) suggestion of designing the system to be
idempotent is fairly hard, depending on the system, but on second
thoughts, maybe a couple of points may help

a. There should be two hidden form elements (or cookie elements)
(1) Session ID, (2) Sequence Number. Sequence numbers start from 0 or
1 per session. Session ID should never repeat (ever), so maybe you
want the date to be part of the session ID.
b. Design the database (or whatever) to have a primary key of
SessionID-SeqNo
c. Have some Insert-Only-If-Not-Exist logic - possibly with an
error if what you're trying to insert the second time has the same
primary key but other fields are different to the first insertion
(probable if it's a BACK button job).
d. The logic then blithly generates the response, after the
Insert-Only-If-Not-Exist logic (whether the request was duplicate or
not).

The only issue remaining is if generating a response is "expensive".
This will be so if it takes a lot of server load, or if the server has
to pull some data from an external data source on a pay-per-pull basis.

In that case, you probably want the response to be a trivial page
saying "Your report/request is being processed. _View_report_now_",
with a background process generating a response, and storing it in,
say, a database field. It's fairly easy for the background process to
ignore duplicate requests.

Security Note: You want to ensure that nobody can fake the session ID,
especially one that was in use a few days ago - so probably make
session IDs unuseable if the user has logged off, or was timed out.

Nick Bishop.