socketServer questions

[rbt]

I have written a python socketServer program and I have a few questions
that I hope the group can answer... here is a simple version of the
server:

class tr_handler(SocketServer.StreamRequestHandler):

def handle(self):

data = self.rfile.readline(300)
data = str.strip(data)
bytes = str(len(data))

public_ip = self.client_address[0]

serv_date = time.strftime('%Y-%m-%d', time.localtime())
serv_time = time.strftime('%H:%M:%S', time.localtime())

# Note that 'data; comes from the client.
fp = file('/home/rbt/Desktop/tr_report.txt', 'a')
fp.write(data+"\t"+serv_date+"\t"+serv_time+"\t"+p ublic_ip+"\t"+bytes+"\n")
fp.close()

if __name__=='__main__':
server = SocketServer.TCPServer( ('', 55503), tr_handler)
server.serve_forever()

---------------------------------------

1. Do I need to use threads to handle requests, if so, how would I incorporate them?
The clients are light and fast never sending more than 270 bytes of data and never connecting
for more than 10 seconds at a time. There are currently 500 clients and potentially there could be
a few thousand... how high does the current version scale?

2. What's the proper way to handle server exceptions (server stops, fails to run at boot, etc.)?

3. How do I keep people from tampering with the server? The clients send strings of data to the
server. All the strings start with x and end with y and have z in the middle. Is requiring x at
the front and y at the back and z someplace in the middle enough to keep people out? I'm open to
suggestions.

Thanks!
rbt

[Paul Rubin]

rbt <rb*@athop1.ath.vt.edu> writes:

1. Do I need to use threads to handle requests, if so, how would I
incorporate them? The clients are light and fast never sending more
than 270 bytes of data and never connecting for more than 10 seconds
at a time. There are currently 500 clients and potentially there
could be a few thousand... how high does the current version scale?
The way it's written now, the server reads a single request, writes
some stuff to the log, and closes the connection. It can't handle
multiple requests simultaneously, but that's ok, no connection stays
open for very long. If you want to have longer-running connections
open simultaneously, you need some type of concurrency such as threads.
But then you have to write the code differently, to serialize the
log recording.

You probably should get a copy of "Python Cookbook" which explains the
basics of multi-threaded programming, if you have to ask a question like that. 2. What's the proper way to handle server exceptions (server stops,
fails to run at boot, etc.)? 3. How do I keep people from tampering with the server? The clients
send strings of data to the server. All the strings start with x and
end with y and have z in the middle. Is requiring x at the front and
y at the back and z someplace in the middle enough to keep people
out? I'm open to suggestions.

It only keeps them out if they don't know to use that x..y..z pattern
and maybe not even then. Get a copy of "Security Engineering" by
Ross Anderson to have an idea of what you're dealing with, especially
if your server controls something valuable.

[Christopher Subich]

Paul Rubin wrote:

rbt <rb*@athop1.ath.vt.edu> writes:

1. Do I need to use threads to handle requests, if so, how would I
incorporate them? The clients are light and fast never sending more
than 270 bytes of data and never connecting for more than 10 seconds
at a time. There are currently 500 clients and potentially there
could be a few thousand... how high does the current version scale?

open for very long. If you want to have longer-running connections
open simultaneously, you need some type of concurrency such as threads.
But then you have to write the code differently, to serialize the
log recording.

You probably should get a copy of "Python Cookbook" which explains the
basics of multi-threaded programming, if you have to ask a question

Or take a look at non-threaded ways of doing non-blocking IO; I've
personally used the Twisted libraries and they work decently without
manual thread overhead [indeed, the default reactor uses select, and
there's a version for 'nix systems that uses poll].

Either way will work, it just depends on how deeply you want to
integrate the network functionality into the code. As someone else said
(paraphrased, an apologies for stealing the quote; a Google search isn't
bringing it up), "You don't use Twisted, you provide Twisted callbacks
to use you."

[rbt]

On Fri, 2005-10-07 at 09:17 -0700, Paul Rubinhttp: wrote:

3. How do I keep people from tampering with the server? The clients
send strings of data to the server. All the strings start with x and
end with y and have z in the middle. Is requiring x at the front and
y at the back and z someplace in the middle enough to keep people
out? I'm open to suggestions.

It only keeps them out if they don't know to use that x..y..z pattern
and maybe not even then. Get a copy of "Security Engineering" by
Ross Anderson to have an idea of what you're dealing with, especially
if your server controls something valuable.

The server just logs data, nothing else. It's not private or important
data... just sys admin type stuff (ip, mac addy, etc.). I just don't
want some script kiddie discovering it and trying to 'hack' it. By doing
so, they'd fill the log up with crap. So, If the data doesn't contain x,
y, and z and if the data is too big or too small, I record it to a
'tamper' log and tell the leet hacker to 'go away'.

[Paul Rubin]

rbt <rb*@athop1.ath.vt.edu> writes:

The server just logs data, nothing else. It's not private or important
data... just sys admin type stuff (ip, mac addy, etc.). I just don't
want some script kiddie discovering it and trying to 'hack' it. By doing
so, they'd fill the log up with crap. So, If the data doesn't contain x,
y, and z and if the data is too big or too small, I record it to a
'tamper' log and tell the leet hacker to 'go away'.

Well, rather than this x,y,z stuff, it's best to do it properly and
authenticate the records with the hmac module.

[rbt]

On Fri, 2005-10-07 at 15:07 -0700, Paul Rubinhttp: wrote:

rbt <rb*@athop1.ath.vt.edu> writes:

The server just logs data, nothing else. It's not private or important
data... just sys admin type stuff (ip, mac addy, etc.). I just don't
want some script kiddie discovering it and trying to 'hack' it. By doing
so, they'd fill the log up with crap. So, If the data doesn't contain x,
y, and z and if the data is too big or too small, I record it to a
'tamper' log and tell the leet hacker to 'go away'.

Well, rather than this x,y,z stuff, it's best to do it properly and
authenticate the records with the hmac module.

Off-topic here, but you've caused me to have a thought... Can hmac be
used on untrusted clients? Clients that may fall into the wrong hands?
How would one handle message verification when one cannot trust the
client? What is there besides hmac? Thanks, rbt

[Tim Williams (gmail)]

On 07/10/05, rbt <rb*@athop1.ath.vt.edu> wrote:

I have written a python socketServer program and I have a few questions

This is a multithreaded non-blocking version of your server (not
tested), with a basic attempt to hande errors.

from socket import *
from SocketServer import *
import time, threading, sys

class tr_server(ThreadingMixIn, TCPServer):
def some_function(self):
pass

class tr_handler(StreamRequestHandler):
global write_to_file, file_path

def handle(self):
print "Current Connection count:", threading.activeCount() -1
public_ip = self.client_address[0]
serv_date = time.strftime('%Y-%m-%d', time.localtime())
serv_time = time.strftime('%H:%M:%S', time.localtime())

try:
data = self.rfile.readline(300)
data = str.strip(data)
bytes = str(len(data))
# Note that 'data; comes from the client.
fp = file('/home/rbt/Desktop/tr_report.txt', 'a')
fp.write(data+"\t"+serv_date+"\t"+serv_time+"\t"+p ublic_ip+"\t"+bytes+"\n")
fp.close()
except:
print "unknown error", sys.exc_info()[0]

def StartServer():
setdefaulttimeout( 30 ) # timeout incoming connections
server = tr_server(('', 55503 ), tr_handler)
server.serve_forever()

if __name__ == '__main__':
StartServer()

Consider putting the writing to file function in its own class or
thread. Then opening the file once and appending the data to it from
each connection. Yoou would need to use fp.flush() after each
fp.write() so that the data survives a program fail.

HTH :)

[Paul Rubin]

rbt <rb*@athop1.ath.vt.edu> writes:

Off-topic here, but you've caused me to have a thought... Can hmac be
used on untrusted clients? Clients that may fall into the wrong hands?
How would one handle message verification when one cannot trust the
client? What is there besides hmac? Thanks, rbt

I don't understand the question. HMAC requires that both ends share a
secret key; does that help? What do you mean by verification? Do you
mean you want to make sure that's really Bob logging into your
computer, even when Bob might have intentionally given his password to
someone else? It sounds like you want something like DRM. What
exactly are you trying to do?

[rbt]

On Sat, 2005-10-08 at 14:09 -0700, Paul Rubinhttp: wrote:

rbt <rb*@athop1.ath.vt.edu> writes:

Off-topic here, but you've caused me to have a thought... Can hmac be
used on untrusted clients? Clients that may fall into the wrong hands?
How would one handle message verification when one cannot trust the
client? What is there besides hmac? Thanks, rbt
I don't understand the question. HMAC requires that both ends share a
secret key; does that help?

That's what I don't get. If both sides have the key... how can it be
'secret'? All one would have to do is look at the code on any of the
clients and they'd then know everything, right?
What do you mean by verification?

I'm trying to keep script kiddies from tampering with a socket server. I
want the server to only load a valid or verified string into its log
database and to discard everything else.

Strings could come to the socket server from anywhere on the Net from
any machine. This is outside my control. What is there to prevent a
knowledgeable person from finding the py code on a client computer,
understanding it and then being able to forge a string that the server
will accept?

Does that make sense?

[Paul Rubin]

rbt <rb*@athop1.ath.vt.edu> writes:

I don't understand the question. HMAC requires that both ends share a
secret key; does that help?
That's what I don't get. If both sides have the key... how can it be
'secret'? All one would have to do is look at the code on any of the
clients and they'd then know everything, right?

Yes, clients have to keep the key secure.

What do you mean by verification?

I'm trying to keep script kiddies from tampering with a socket server. I
want the server to only load a valid or verified string into its log
database and to discard everything else.

If the clients can keep a secret key secure, then use hmac. Note that
if there's lots of clients, they shouldn't all use the same secret key.
Instead, for client #i, let that client's key be something like
hmac(your_big_secret, str(i)).digest()
and the client would send #i as part of the string. You'd use
#i to recompute the client's key and then use that derived key to
verify the string. This is called "key derivation" or "key
diversification". If an attacker gets hold of that client's key and
starts hosing you, you can disable that key without affecting the
other ones. (The client is issued only the derived key and never sees
the big secret).
Strings could come to the socket server from anywhere on the Net from
any machine. This is outside my control. What is there to prevent a
knowledgeable person from finding the py code on a client computer,
understanding it and then being able to forge a string that the server
will accept?

Yes, if you're concerned about insecure clients, you have a much more
complex problem. But your x..z..y scheme is far worse than hmac.
Once the attacker figures that out, there's no security at all.

What is the actual application, if you can say? Depending on the
environment and constraints, various approaches are possible.

[rbt]

On Mon, 2005-10-10 at 05:54 -0700, Paul Rubinhttp: wrote:

rbt <rb*@athop1.ath.vt.edu> writes:

I don't understand the question. HMAC requires that both ends share a
secret key; does that help?
That's what I don't get. If both sides have the key... how can it be
'secret'? All one would have to do is look at the code on any of the
clients and they'd then know everything, right?

Yes, clients have to keep the key secure.

What do you mean by verification?

I'm trying to keep script kiddies from tampering with a socket server. I
want the server to only load a valid or verified string into its log
database and to discard everything else.

If the clients can keep a secret key secure, then use hmac. Note that
if there's lots of clients, they shouldn't all use the same secret key.
Instead, for client #i, let that client's key be something like
hmac(your_big_secret, str(i)).digest()
and the client would send #i as part of the string.

How is this different from sending a pre-defined string from the client
that the server knows the md5 hash of? The clients know the string, the
server knows the hash of that string.

Also, could this not be done both ways? So that, if an attacker figures
out the string he's supposed to send from a client to the server (which
he could easily do). He could not easily figure out the string the
server should send back as all he would have is the hash of that string.

So, before the actual data is sent from the client to the server. The
client would send it's secret string that the server would verify and
then if that worked, the server would send its own secret string that
the client must verify. We'd have two secret strings instead of one.

You'd use
#i to recompute the client's key and then use that derived key to
verify the string. This is called "key derivation" or "key
diversification". If an attacker gets hold of that client's key and
starts hosing you, you can disable that key without affecting the
other ones. (The client is issued only the derived key and never sees
the big secret).
This is interesting. I didn't know that was possible.

Strings could come to the socket server from anywhere on the Net from
any machine. This is outside my control. What is there to prevent a
knowledgeable person from finding the py code on a client computer,
understanding it and then being able to forge a string that the server
will accept?
Yes, if you're concerned about insecure clients, you have a much more
complex problem. But your x..z..y scheme is far worse than hmac.
Once the attacker figures that out, there's no security at all.

I dropped the x,y,z scheme after your first response ;)

What is the actual application, if you can say? Depending on the
environment and constraints, various approaches are possible.

Nothing important. It just logs network data. It's an anti-theft program
for laptops that phones home data like this: public and private IP(s),
MAC addy, date, time, etc. Maybe I'm putting too much thought into it.
Python encourages good design and I try to follow that encouragement
when coding... even for trivial things such as this.

[Paul Rubin]

rbt <rb*@athop1.ath.vt.edu> writes:

Instead, for client #i, let that client's key be something like
hmac(your_big_secret, str(i)).digest()
and the client would send #i as part of the string.
How is this different from sending a pre-defined string from the client
that the server knows the md5 hash of? The clients know the string, the
server knows the hash of that string.

I'm confused, I don't understand what that md5 whatever would do for you.
I'm assuming the server is secure and the clients are less secure.
Also, could this not be done both ways? So that, if an attacker figures
out the string he's supposed to send from a client to the server (which
he could easily do). He could not easily figure out the string the
server should send back as all he would have is the hash of that string.
I'm still confused, are you now trying to prevent the client from
accepting bogus stuff from the server? You can do that with a digital
signature and not need client-side secrets. But I still don't see
much gain from that.
So, before the actual data is sent from the client to the server. The
client would send it's secret string that the server would verify and
then if that worked, the server would send its own secret string that
the client must verify. We'd have two secret strings instead of one.
If the client is insecure, an attacker who gets control of it will get
all the secrets in it, so it doesn't matter if there's one or several.
Nothing important. It just logs network data. It's an anti-theft program
for laptops that phones home data like this: public and private IP(s),

Yeah, if a laptop is stolen and the thief is sophisticated enough to
go finding the authentication keys inside some binary, s/he also won't
be dumb enough to leave the program active. Those programs are just
for dumb thieves (of whom there are a lot) or their customers, who
simply plug in the laptop and start using it, without concern about
what software is on it. If they have any sense they'll do a total
reinstall.

But anyway, yeah, people who enroll in the system should get a
customer number. Then you can derive an auth key from the customer
number as described earlier.

[rbt]

On Mon, 2005-10-10 at 07:46 -0700, Paul Rubinhttp: wrote:

rbt <rb*@athop1.ath.vt.edu> writes:

Instead, for client #i, let that client's key be something like
hmac(your_big_secret, str(i)).digest()
and the client would send #i as part of the string.

How is this different from sending a pre-defined string from the client
that the server knows the md5 hash of? The clients know the string, the
server knows the hash of that string.

I'm confused, I don't understand what that md5 whatever would do for you.
I'm assuming the server is secure and the clients are less secure.

Also, could this not be done both ways? So that, if an attacker figures
out the string he's supposed to send from a client to the server (which
he could easily do). He could not easily figure out the string the
server should send back as all he would have is the hash of that string.

I'm still confused

OK, we'll leave it at that and just accept that we're from different
planets ;) Thanks for the help.