Functions that aren't thread-safe can indeed be used from multiple threads and they will often appear to work fine. The problem is there are certain timing windows where execution in one thread damages another. The program fails sporadically; and the failure mode may be subtle or catastrophic.
If you're lucky you'll see some of these failures while you're testing . If you're not lucky, the failures won't start until you're demonstrating your code to somebody important (boss, teacher, customer,...).
Debugging one of these problems is typically very difficult. After all, there is no flaw in your program; the problem lies in a library function that you don't have the source code for.
I'm aware of the problems of not mutexing things and so on and so forth, but what I'm worried here is about the specific issue:
"Is it possible to work on the same file descriptor (for a socket, if that's relevant) from several threads at the same time?"
If not, why not? How can I overcome this?
My problem is that I have to implement bidirectional communication, without no end being a master. I.e., each of the sides can start sending at any time, so I can't afford having a thread stuck at read(), since it won't be able to write() until read() is done (and read() will remain blocked most of the time).
I'm also aware that I can fcntl() the read so that it does not block on call. But that would mean busy-checking the input, and I have 23 threads more in my app demanding execution time.