Multi-user Password Database Solutions?

[beporter]

First time poster here! Let me get straight to it...

I'm currently not in a position that lets me interact with other developers face to face on a regular basis, and I am in need of some "round table" discussion on a project that has be given to me. Let me describe the functional requirements I have in as general terms as possible first:

1) The task charged to me is to investigate, and if possible implement, a system for storing a great many passwords as securely as possible.

2) The purpose of each password does *not* need to be stored in the datastore. There does not need to be a URL, a username, a description or any other associated information in the datastore. The only thing I do need is some kind of unique identifier which could be tied into other data-tracking systems by referencing the ID for a given password in the system. I.e.: sending someone an email instructing them to, "Go look up password number 432 for this." In other words; my current plan is to make the password storage as agnostic as possible. I can implement the tracking of notes, usernames, and purposes in any number of ways: posting in a wiki, writing them down on paper, or saving a Word document. The only thing I really need to lock down good and tight are the passwords themselves.

3) The system must allow multiple, concurrent user "logins" to access/update the stored passwords. There is no need for per-user permissions for subsets of the stored information: any user in the system is allowed to access ALL information stored inside.

4) A high priority is placed on logging all actions performed by each user: including when a certain password is viewed or changed, by which user, and any other available information pertainent to the action.

5) The interface for accessing the passwords (if it is different from the software used for managing the data) should ideally not be locked to a given operating system (meaning I'll likely reject a pre-existing application that only runs in Windows, for example).

6) The system should also preferably be resistant to mass-viewing: it should be impossible (using the given interface at least) to view or modify more than one password at a time, and as difficult as possible to obtain a copy of the datastore file(s) themselves.

7) The last request is that the datastore as a whole must be backed up regularly for redundancy, and hence, the backup format/system utilized must also be as secure as possible.

There it is. This is not at all an easy "problem" to solve, and repeated and varied Googlings have not turned up anything close enough to what I'm looking for. My current line of thinking is not entirely satisfying to me and I need to see if somebody more knowledgeable than myself naturally comes up with a solution I would not have considered-- especially if there's a possibility I'm grossly over-thinking this entire thing.

Anyway, I've tried to keep my description as generic as possible to prevent biasing the reader's own thought process. I'd of course like to hear any thoughts the community has on this. I'd be happy to entertain any existing applications (FOSS or otherwise), as well as suggestions for a "from scratch" implementation. If anyone has questions they need to ask to clarify the problem, I'd be only too happy to answer them.

I'm also eager to share my current ideas for implementing this, but again, I want to see what unique suggestions everyone can come up with first before artifically locking the discussion into any particular category or programming language.

Lastly, I apologize if this is not the correct board to post this under , but as I said I am not currently restricting this project to being a "database project" or a "C++ project", etc. The _method_ of implementation is not as important as the security of the resulting system as a whole. I'm hoping the power of 39,000 other minds can help come up with a more secure, reliable system than just my one can.

Thanks for you time!

[Killer42]

First time poster here! Let me get straight to it...
etc...

Hi.

I'd just like to make one small technical point - or perhaps it would be better termed "policy" than technical. As a general rule, you should avoid storing passwords unencrypted. In any location which might conceivably be compromised, you should store passwords only in an encrypted form. To check it, you take the password entered by the user, encrypt that, and then check it against the database.

[beporter]

As a general rule, you should avoid storing passwords unencrypted. In any location which might conceivably be compromised, you should store passwords only in an encrypted form. To check it, you take the password entered by the user, encrypt that, and then check it against the database.

A perfectly acceptable and reasonable point, however it unfortunately neglects the purpose the database. The passwords themselves need to be recoverable. That is, in fact, the whole point of the database: to securely store a number of passwords in a single location so that they can be retrieved and used later. I'm thinking a lot like the Mac OS X Keychain (the description here is better than Apple's if you are unfamiliar with the technology), except with distributed access and not platform dependent.

Does anybody know of any existing packages, or have any good suggestions for ways to implement this?

Since there's been so little interest, I might as well go ahead and throw my own brainstorming out there. Keep in mind this is heavily influenced by the tools I'm most comfortable using. Currently my thinking is to use a MySQL database with a very simple table layout:

CREATE TABLE `pwdb` (
  `id` int(11) NOT NULL auto_increment,
  `pwc` tinyblob,
  PRIMARY KEY  (`id`)
) TYPE=MyISAM

We'll need a second table to track login rights, again very simple:

CREATE TABLE `login` (
  `id` int(11) NOT NULL auto_increment,
  `username` varchar(255) NOT NULL default '',
  `password` tinyblob NOT NULL,
  PRIMARY KEY  (`id`)
) TYPE=MyISAM

Obviously lock the database down to access from localhost only. On top of this, write a web-based script (PHP/Perl/Whatever) to access it. I can run all of this on an internal webserver and block public access both from Apache and using the network config/firewall. We can use VPN to get into the network to access the application.

Now as I said originally, there are a lot of gotchas to watch for in this setup and without public scrutiny I'm likely to miss at least a couple of them.

Anyone have any feedback?

[beporter]

Please also keep in mind that the client requesting this "solution" is currently storing their passwords in cleartext in an Excel file hosted on an open network share. In other words, they are in critical need of a huge increase in security. The system doesn't have to be perfect, but it has to be a couple orders of magnitude better than what they have currently.

[beporter]

Well, 10 days and only one response, which unfortunately wasn't even so helpful. I can't believe nobody has any suggestions at all!

Thanks to Killer for his input.

I doubt I'll come back to thescripts.com for any future discussion.

[Killer42]

Well, 10 days and only one response, which unfortunately wasn't even so helpful. I can't believe nobody has any suggestions at all!

Thanks to Killer for his input.

I doubt I'll come back to thescripts.com for any future discussion.

Sorry you didn't get much of a response here in the Access forum.

If you're still around, I'd suggest you try this one in the "Database Developers lounge". Description: Chat about latest database news, trends and technologies. Network and chat with other database developers.

It's a bit odd, really - just about any post here usually develops into a lively discussion, and produces at least some kind of results, or at least ideas.

[Killer42]

Hi folks: I recently switched my website from another provider to ...

Yeah thanks for that, Alex (or Peter, or whoever you are). Perhaps not strictly relevant, though?

Oh, and sorry beporter, I thought your post was in the Access forum - got myself a bit mixed up.

[beporter]

Thanks yet again Killer. I've posted a redirect thread in the Database Lounge. Hopefully that might bring a couple new eyes here and generate some discussion. I won't give up hope yet, and I genuinely appreciate your help.

It's a shame that so many minds have to be so segregated. I mean, it makes sense to have topical boards for certain things, so that people who have very specific questions know the best place to ask them, but what about the meta-questions that span MANY programming topics, like this one? Databases, Web Programming, System Programming, Security, Encryption, Networking, Best Practices... There is no board here that seems appropriate for all of those topics.

In fact, I my very first choice would have been a "Security" board, which I don't seem to see represented at all! Maybe it's just me, but that seems like an oversight of a rather large proportion.

[Killer42]

Thanks yet again Killer. I've posted a redirect thread in the Database Lounge. Hopefully that might bring a couple new eyes here and generate some discussion. I won't give up hope yet, and I genuinely appreciate your help.

It's a shame that so many minds have to be so segregated. I mean, it makes sense to have topical boards for certain things, so that people who have very specific questions know the best place to ask them, but what about the meta-questions that span MANY programming topics, like this one? Databases, Web Programming, System Programming, Security, Encryption, Networking, Best Practices... There is no board here that seems appropriate for all of those topics.

In fact, I my very first choice would have been a "Security" board, which I don't seem to see represented at all! Maybe it's just me, but that seems like an oversight of a rather large proportion.

Yeah, you could be right. Still, I suppose it takes time to fine-tune these things. I know "thescripts" is still undergoing a lot of change and refinement, things moving around and so on. KUB365 runs the place, I believe, so feel free to make suggestions.

I suppose one problem with a very general area is that people tend to look at the areas that interest them the most. For instance, I mostly scan the VB and Access forums. I also occasionally drop into the Programmer's Lounge and Community Lounge, but there's rarely anything there, and even more rarely anything that I find interesting or that I can help with. We have to work within our limits.

Now if we could get paid for providing support like this, I'm sure you'd see a lot more interest. :)

Anyway, I hope you see a more helpful response in the DB lounge. Assuming anyone reads it, that would be my guess as to the best place for it. It might be worth dropping a link into the Access forum, as well (I know, that's where I thought you were originally). But there are a few people there who seem to know a lot about databases.

Good luck!

[MMcCarthy]

I'm going to post a number of redirects for you in the Access, VB, Java, C++/C and Web forums.

Our experts hang out more in the technical forums than the lounges and should generate a lively discussion.

Mary

Thanks yet again Killer. I've posted a redirect thread in the Database Lounge. Hopefully that might bring a couple new eyes here and generate some discussion. I won't give up hope yet, and I genuinely appreciate your help.

It's a shame that so many minds have to be so segregated. I mean, it makes sense to have topical boards for certain things, so that people who have very specific questions know the best place to ask them, but what about the meta-questions that span MANY programming topics, like this one? Databases, Web Programming, System Programming, Security, Encryption, Networking, Best Practices... There is no board here that seems appropriate for all of those topics.

In fact, I my very first choice would have been a "Security" board, which I don't seem to see represented at all! Maybe it's just me, but that seems like an oversight of a rather large proportion.

[Banfa]

Here are some security points

1. Just because you need to retrieve the actual passwords doesn't mean that you shouldn't encrypt them to put them in the database. Some enyryption methods (MD5 for instance) are 1 way and are only used to authenticate a given password, you encrypt the password and see if it matches the encrypted passwords stored in the database. However if you use a 2 way encryption method (i.e. 1 can can encrypt and then decrypt the data) you can store the data encrypted and dycrypt it for viewing editing.

2. If you are going to have a user table that contains logon passwords it will need to be just as secure as the password table.

3. Now it sounds like this may not apply because you are inside a company firewall so there should be no external access but access through the webserver will send the data in the clear (unless you use a secure server). That means anyone sniffing the network (just retrieving all packets) will have access to unencrypted version of any passwords currently being retrieved. The VPN should handle encryption for you from outside the company.

4. You have not made it clear what the percieved security threat that this is trying to combat is. If the data is already inside the company firewall who are you trying to prevent access from? Or is this more about allowing multiple people within the company read/write access to the data concurrently rather than securing it from a percieved threat?

[NeoPa]

A couple of points.
1) I understand why encryption may be a problem, but a simple bitwise NOT might give you a little higher security and allow your interface easy access.
2) Don't know much about web programming myself, but would think that's the best way to get platform independent code working quickly and easily.
3) I think most RDBMS systems should be able to handle your main requirements - backup; availability etc. Access would find it difficult to remain available while being backed up.

I hope you appreciate that, while we try to keep on top of questions here and answer technical problems promptly, as your thread is a little more involved, devoting large amounts of time to it is something we have to watch out for, with a view keeping balanced and covering all requests.

Lastly, let me commend you on the rare clarity with which you expressed your issue. Without that I'm afraid I would have passed on to another question as it is quite complex. I hope I've helped some. If you have further questions in this thread, please don't feel ignored if responses aren't immediate - sometimes people just don't know the answer - and maybe the one person who can help is particularly busy or away on holiday.

BTW I hadn't seen Banfa's response when I formulated this post.
It looks like his security help is better than mine anyway :(.

[alsutton]

Apologies for coming to the thread late.

You should probably take a look at the enterprise password safe at http://www.enterprise-password-safe.com/, I think it has all the features you're looking for, and buying in would save you the development, support, and maintenance time of going down the route of an in-house solution.

If it doesn't I work for the company developing it and I know we add customer requests to the development plan so just get in touch.

Al.

[AricC]

Just saw this thread. What kind of platforms do you have to work with, ASP, ASP.Net, ASP 2.0, PHP, Perl etc... There are ton's of Login/Password examples available on the web that I'm sure would do plenty for what you want; I would hesitate on buying anything unless you really don't want to mess with doing a little coding.

[beporter]

Thank you very much for all of your feedback!

I'm afraid this is just one of many projects currently on my plate, and I apologize profusely if it takes me a little while to reply. I am anxious to sit down over the long weekend coming up and respond to all of your input, but right now I must attend to more pressing matters. (Clients are clients.)

Again, thank you all for your input, and I hope that this discussion will eventually turn into something (software, instructions, tips) that will be useful to others!

Please be patient with me, and expect some replies by next week!

[beporter]

1. Just because you need to retrieve the actual passwords doesn't mean that you shouldn't encrypt them to put them in the database. ... However if you use a 2 way encryption method (i.e. 1 can can encrypt and then decrypt the data) you can store the data encrypted and dycrypt it for viewing editing.

Of course. Though if somebody does obtain a copy of the passwords table, the difference between whether it's encrypted or not will make very little difference-- really it will just be a matter of how much time it takes before they crack it. Computing power is cheap enough now that you might as well have given them an unencrypted copy. Somebody tell me if I'm off base.

2. If you are going to have a user table that contains logon passwords it will need to be just as secure as the password table.

Another good point. This will always be a potential weak point in any system that humans have to access. It all comes down to password strength. However, the trick here is that we can control how strong passwords must be for new user accounts.

That means anyone sniffing the network (just retrieving all packets) will have access to unencrypted version of any passwords currently being retrieved.

Good point. I don't think anybody in the office is capable of that level of sophistication, but it can't hurt to be careful anyway.

If the data is already inside the company firewall who are you trying to prevent access from? Or is this more about allowing multiple people within the company read/write access to the data concurrently rather than securing it from a percieved threat?

Yes, the database IS meant to facilitate concurrent access from multiple users, but there are a few threats I've identified partly in conjunction with those users. First, as I mentioned earlier, this system is meant to allow company employees to access a list of shared passwords without making those passwords:

1) easy for "outsiders" to obtain. Currently there is an Excel spreadsheet on a non-password protected Samba share on a network where the wireless access is protected by 64bit WEP. It would not be difficult at all for somebody to gain access to ALL the files on the server.

2) too easy for employees to obtain a complete copy of the list. This is really only an issue if an employee leaves the company or were to become disgruntled. To copy this Excel file to a USB drive and walk out the door with it would again be easy to do. Because of the nature of the services the company works with that the passwords protect, this could be a Very Bad Thing(TM). Changing all of these passwords quickly when an employee quits or is terminated would be nearly impossible, making the services protected by these passwords vulnerable to the ex-employee for a significant period of time. Now, there's nothing stopping an employee from secretly starting and maintaining their own list of passwords, but I can't think of any way to counter that no matter what you do. Hire honest people I guess.

My real goal is as I originally stated: to make obtaining the complete list of passwords (and their uses) difficult. I must concede to the fact that it will be impossible to secure ALL of the passwords from ALL the threats at the same time, but I need to do better than an Excel file on an open sharepoint.

[NeoPa]

Of course. Though if somebody does obtain a copy of the passwords table, the difference between whether it's encrypted or not will make very little difference-- really it will just be a matter of how much time it takes before they crack it. Computing power is cheap enough now that you might as well have given them an unencrypted copy. Somebody tell me if I'm off base.

You're off base.

Well - a little anyway.
If this is to protect against employees (primarily) then the average employee will be confused / put off by a list of passwords in garbled form.
As I understand it, you have no clue in the table as to what it's a password for, nor would you publish the actual method of encryption. I think this would help your situation overall, but it couldn't, of itself, answer all your requirements.

[beporter]

A couple of points.
1) I understand why encryption may be a problem, but a simple bitwise NOT might give you a little higher security and allow your interface easy access.

I'm not sure this would present a significant stumbling block for an attacker competent enough to obtain a dump of the MySQL database to begin with, but as I said in my previous post, it can't hurt.

2) Don't know much about web programming myself, but would think that's the best way to get platform independent code working quickly and easily.

That was my thinking as well. PHP, Perl, and a whole range of others are perfectly acceptable.

3) I think most RDBMS systems should be able to handle your main requirements - backup; availability etc. Access would find it difficult to remain available while being backed up.

Again I agree. MySQL is a piece of cake.

I hope you appreciate that, while we try to keep on top of questions here and answer technical problems promptly, as your thread is a little more involved, devoting large amounts of time to it is something we have to watch out for, with a view keeping balanced and covering all requests.

Yes, I understand. I certainly don't want to hog anyone's time, and I have no delusions that my request is any more important than anyone else's. Please understand that I was originally disappointed at the turn out. Having no prior experience with this site, my expectations for responses from a perceived pool of 40,000 members was perhaps unrealistically high.

Lastly, let me commend you on the rare clarity with which you expressed your issue. Without that I'm afraid I would have passed on to another question as it is quite complex.

When all you have is text to communicate with, what choice does one have than to be explicitly clear? :-)

I hope I've helped some. If you have further questions in this thread, please don't feel ignored if responses aren't immediate - sometimes people just don't know the answer - and maybe the one person who can help is particularly busy or away on holiday.

Very much so, in fact. As I said, I wanted to harness the power of many bright minds, and the feedback generated is most helpful.

It looks like his security help is better than mine anyway :(.

There's no such thing as "too much security help" and I will always listen to any and all suggestions-- especially when it comes to computer security. It's an area that is grossly overlooked by far too many programmers. (Take for example the fact that there is no board on this site dedicated to it!)

[NeoPa]

When all you have is text to communicate with, what choice does one have than to be explicitly clear? :-)

In answer to that, just look around you.
Clarity is a rare commodity - and that's not restricted to foreign posters whose first language is other than English.

There's no such thing as "too much security help" and I will always listen to any and all suggestions-- especially when it comes to computer security. It's an area that is grossly overlooked by far too many programmers. (Take for example the fact that there is no board on this site dedicated to it!)

For there to be a Security section set up, there would have to be the interest in the topic as well as enough experts to provide answers.
You can take it as read that there is neither. From your obvious position of experience I'd be surprised if you didn't understand the main reason why.

Thank you for your replies and keeping us updated as to your thinking. I don't want to quote every section but you can believe all was read with interest.

[MMcCarthy]

There's no such thing as "too much security help" and I will always listen to any and all suggestions-- especially when it comes to computer security. It's an area that is grossly overlooked by far too many programmers. (Take for example the fact that there is no board on this site dedicated to it!)

I agree with the point.

This is a fairly new site, as these things go and forums are dependent on enough experts being available to answer questions on the specialities.

I do think though that a site on Network securtiy would be a valuable addition and as this site is currently under reconstruction I will suggest that it go on the to-do list.

[beporter]

You should probably take a look at the enterprise password safe at http://www.enterprise-password-safe.com/

Wow, that's darn near exactly what I am looking for. The system is RDBMS-based, which abstracts data storage and backup, it's accessible from the web so it's highly cross-platform, it has a high level of self-auditing, the storage of the passwords is done in a highly secure fashion, and it allows multiple concurrent user logins. It has extras that I wouldn't need in this case, such as per-user restrictions, but those can be ignored of course. I think the price might be the only sticking point.

$2,000+ USD (for 5 users, a year of "free" upgrades and 72 hour "priority" email support) is probably more than this company is going to be willing to spend, unfortunately. I do very much like the frequency with which the package seems to be updated (if the changelog is any indication), but it appears that the product was initially released only 7 months ago at this point. Despite the activity, even if it was within a more reasonable price range I would probably want to wait for others to provide more "real world" testing before investing myself.

Also, I want to be clear that I'm not saying $2,000 is an unreasonable amount given the quality of the product: only that it exceeds what I expect to be able to get this given company to spend on protecting an asset they are currently grossly undervaluing to begin with. I don't want to imply that I thought the price was outrageous, only that my client probably will.

Thank you for pointing this software out, Al!

[beporter]

Just saw this thread. What kind of platforms do you have to work with, ASP, ASP.Net, ASP 2.0, PHP, Perl etc... There are ton's of Login/Password examples available on the web that I'm sure would do plenty for what you want; I would hesitate on buying anything unless you really don't want to mess with doing a little coding.

I would prefer to avoid any Microsoft-centric technologies if a homebrew solutions is in order. Anything more "openly" available is fine though. I can handle just about any programming language, but Perl or PHP would probably be my first picks unless I was given a reason to use something else.

I've looked at quite a few open source solutions that already exists, such as KeePass, w3pw, Universal Password Manager, PasswordChain and pretty much everything here. None of these are particularly satisfying for various reasons. The most common reason is the package being single-user-centric.

This is why I was starting to think I'd have to roll my own software--which I have no problem at all doing if it's going to be the "best" solution. However, if that's the case I'd like to think that I know just enough about software security to know I don't know enough about software security to lock my project down thoroughly enough. Hence my posting the problem here. :)

[beporter]

You're off base.

Well - a little anyway.
If this is to protect against employees (primarily) then the average employee will be confused / put off by a list of passwords in garbled form.
As I understand it, you have no clue in the table as to what it's a password for, nor would you publish the actual method of encryption. I think this would help your situation overall, but it couldn't, of itself, answer all your requirements.

Yes, I do need to protect against the employees who are technically less capable, but I also need to protect against an attacker that might be relatively competent. Yes, a garbled password would thwart the average employee, but would give us no extra protection against anyone else. The chances of such an attack are much lower of course, but the consequences are much worse.

However, the important thing to keep in mind is that the overall security of the system is only as good as the weakest component. The chink in the armor, so to speak. I just want to make sure I think about the implications of all of the components involved so as not to miss one, and that's why I thank you all for your help!

[beporter]

So to sum up, here's what it looks like we've come up with:

* Standard LAMP setup on a locked down server (only ports 80/443 open for Apache). Once the system is in place, I can have the owner of the company enter a new root password to help protect the console. Further, we can physically lock the machine up if necessary.

* MySQL database (locked down to only localhost access) with 3 tables:

   passwords{ int:id, string:password }
   users{ int:id, string:username, string:passwordHash }
   logs{ int:id, int:eventCode, int:userID, timestamp:eventTime, string:auxilaryInfo }
 

* Apache locked down to allow connections either from the local subnet only, or specific workstation IPs if necessary.

* PHP script for CRUDing user accounts. (Must enforce strong login passwords for all accounts. Must log all activity.)

* PHP script for user log in, password CRUD, password ID number searching, and single password display. (Also must log all activity.) There must also be a mechanism in place to detect a CURL-like script automatically downloading passwords from the web interface. Perhaps a CAPTCHA system before displaying a password?

* Shell(?) script for performing mysqldump to file, encryption of file, and uploading of encrypted file to backup server. Script can be scheduled through cron to run as often as necessary. The encryption key for the backup files MUST be copied to another secure location, otherwise if the server's hard drive were to die we would not be able to decrypt the backup files.

*All created software should be archived in a safe location in case the server needs to be recreated.


That should about cover the actual implementation, however, there are some auxiliary steps to ensure success:

* Employee training. This is essential to get them to start using the software and for them to recognize the importance of it, not to mention teaching them HOW to use the software.

* Locking down the wifi access. Again essential to [help] prevent unauthorized network access. (This is a good idea whether I implement anything or not!)


I'm doing this too quickly to have remembered everything. What have I missed?

There is at least one issue that's come to mind that I don't know how to address in this setup. The passwords are keyed on an ID number only. Other company documentation will record the purpose of a password, along with the ID number only. Say a Word document that tells employees how to log into their online IRA account for example. The Word document might make reference to the password database: "to log in, look up password # 447 in the database". This works fine for seperating the secret from the intended use, but what if the Word document itself is destroyed or lost? We can easily recreate all of the information from memory EXCEPT for the password ID. We'd have no way of recovering that from the password database alone. Anybody have any thoughts?

[NeoPa]

Perhaps a hidden & encrypted table storing all the links / correlations might work.
Would that be secure enough?

[alsutton]

$2,000+ USD (for 5 users, a year of "free" upgrades and 72 hour "priority" email support) is probably more than this company is going to be willing to spend, unfortunately. I do very much like the frequency with which the package seems to be updated (if the changelog is any indication), but it appears that the product was initially released only 7 months ago at this point. Despite the activity, even if it was within a more reasonable price range I would probably want to wait for others to provide more "real world" testing before investing myself.

Hi beporter,

The product is over 3 years old, The latest version (1.57) is 7 months old and is the most recent, but the product has been developed over the last three and a half years and wen't through a 1.00 -> 1.10 then 1.50 -> 1.57 where it is today with the next version due in Q2 2007.

The free upgrade option wouldn't be suitable for 5 users. You can upgrade on a per-user basis which works out at about 100 USD per release, and there is usually one major release per year (patches as shown in the changlog are free).

The priority support option is really your call, but pretty much all of the support questions we get are from users who have upgraded a part of the server and had problems.

If you want to know more use the contact us form on the site and I'll make sure I pick it up.

Al.

[beporter]

I haven't disappeared: just more sporadic availability.

Thanks for the feedback Al, that's very helpful. I still think the price point might be a sticking point, but it would definitely make it more reasonable with your explanations. It's odd that the default set of purchase options on the website doesn't really work together though.

I'll provide some more feedback as soon as I can!