Atomic line-reading without buffering

[japhy]

Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?

[santosh]

japhy wrote:

Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?

Within Standard C all you can do is use the setvbuf function and
specify no buffering with _IONBF as one of it's arguments. Despite
this, the operating system may still use disk buffers when doing I/O.
To turn off all buffering is system dependant and you'll have to
examine the facilities your OS provides to do so.

The word atomic is inappropriate in the context in which you've used
it. Even when buffering is disabled, I/O is not guaranteed to be an
atomic operation.

[Walter Roberson]

In article <11********************@p10g2000cwp.googlegroups.c om>,
santosh <sa*********@gmail.comwrote:

>japhy wrote:

>Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?

>The word atomic is inappropriate in the context in which you've used
it. Even when buffering is disabled, I/O is not guaranteed to be an
atomic operation.

Though if you make the step over to POSIX, writes below a
system-dependant size are promised to be atomic. POSIX needs
this property because it deals with multiple processes (and
multiple threads in later editions) and it would be a real
nuisance to have the output of one write intermixed with the
output of another write.

--
If you lie to the compiler, it will get its revenge. -- Henry Spencer

[santosh]

Walter Roberson wrote:

In article <11********************@p10g2000cwp.googlegroups.c om>,
santosh <sa*********@gmail.comwrote:

japhy wrote:

Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?

The word atomic is inappropriate in the context in which you've used
it. Even when buffering is disabled, I/O is not guaranteed to be an
atomic operation.

Though if you make the step over to POSIX, writes below a
system-dependant size are promised to be atomic. POSIX needs
this property because it deals with multiple processes (and
multiple threads in later editions) and it would be a real
nuisance to have the output of one write intermixed with the
output of another write.

Yes, thanks, I should probably have mentioned that, but I was thinking
more along the lines of the C standard's assurances with regard to
atomic operations.

[CBFalconer]

santosh wrote:

>

.... snip ...

>
Yes, thanks, I should probably have mentioned that, but I was
thinking more along the lines of the C standard's assurances
with regard to atomic operations.

The C standard runs on a dedicated machine with no processes,
threads, interrupts, etc. and thus has no need of atomic
operations, semaphores, etc.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home.att.net>

[Richard Heathfield]

CBFalconer said:

santosh wrote:

>>

... snip ...

>>
Yes, thanks, I should probably have mentioned that, but I was
thinking more along the lines of the C standard's assurances
with regard to atomic operations.

The C standard runs on a dedicated machine with no processes,
threads, interrupts, etc. and thus has no need of atomic
operations, semaphores, etc.

Almost true. Even if we ignore the C99 floating point stuff as being
irrelevant here, we still must not forget sig_atomic_t, which is
defined to be "the (possibly volatile-qualified) integer type of an
object that can be accessed as an atomic entity, even in the presence
of asynchronous interrupts".

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.

[SM Ryan]

"santosh" <sa*********@gmail.comwrote:
# Walter Roberson wrote:
# In article <11********************@p10g2000cwp.googlegroups.c om>,
# santosh <sa*********@gmail.comwrote:
# japhy wrote:
# Is there a way to read a line (a series of characters ending in a
# newline) from a file (either by descriptor or stream) atomically,
# without buffering additional contents of the file?
# >
# The word atomic is inappropriate in the context in which you've used
# it. Even when buffering is disabled, I/O is not guaranteed to be an
# atomic operation.
# >
# Though if you make the step over to POSIX, writes below a
# system-dependant size are promised to be atomic. POSIX needs
# this property because it deals with multiple processes (and
# multiple threads in later editions) and it would be a real
# nuisance to have the output of one write intermixed with the
# output of another write.
#
# Yes, thanks, I should probably have mentioned that, but I was thinking
# more along the lines of the C standard's assurances with regard to
# atomic operations.
#
#
#

--
SM Ryan http://www.rawbw.com/~wyrmwif/
One of the drawbacks of being a martyr is that you have to die.

[bluejack]

On Mar 2, 2:31 pm, Richard Heathfield <[e-mail]:

Almost true. Even if we ignore the C99 floating point stuff as being
irrelevant here, we still must not forget sig_atomic_t, which is
defined to be "the (possibly volatile-qualified) integer type of an
object that can be accessed as an atomic entity, even in the presence
of asynchronous interrupts".

Just for my own information, does "asynchronous interrupts" include
thread pre-emption? Is this how thread mutexes are implemented? I was
perusing the boost library recently and saw a lot of inline assembly
for mutexes, etc. I presume if sig_atomic_t were thread-safe as well
as signal safe the boost people could have saved a lot of trouble?

Or is that because in something like:

void do_something(void)
{
static sig_atomic_t mx = 0;
start:
if (mx++ == 0) {
/* Wait for mx to be zero */
goto start;
} else {
/* Do some work */
}
}

mx may be atomic, but the test (mx++ == 0) is not guaranteed to be an
atomic operation? (And I presume there's no way to make such a
guarantee in C alone.)

-Bluejack

[santosh]

SM Ryan wrote:

"santosh" <sa*********@gmail.comwrote:
# Walter Roberson wrote:
# In article <11********************@p10g2000cwp.googlegroups.c om>,
# santosh <sa*********@gmail.comwrote:
# japhy wrote:
# Is there a way to read a line (a series of characters ending in a
# newline) from a file (either by descriptor or stream) atomically,
# without buffering additional contents of the file?
# >
# The word atomic is inappropriate in the context in which you've used
# it. Even when buffering is disabled, I/O is not guaranteed to be an
# atomic operation.
# >
# Though if you make the step over to POSIX, writes below a
# system-dependant size are promised to be atomic. POSIX needs
# this property because it deals with multiple processes (and
# multiple threads in later editions) and it would be a real
# nuisance to have the output of one write intermixed with the
# output of another write.
#
# Yes, thanks, I should probably have mentioned that, but I was thinking
# more along the lines of the C standard's assurances with regard to
# atomic operations.
#
#
#

Err, did you have something to say? Your post appears empty of a
reply, (except for the quoted text), but maybe that's a Google Groups
issue?

[SM Ryan]

# Is there a way to read a line (a series of characters ending in a
# newline) from a file (either by descriptor or stream) atomically,
# without buffering additional contents of the file?

Generally once data is transferred from system to program
unless you're explicitly using locks, you lose atomicity.
On systems that transfer chunks of chars and expect the program
to find line structure in that, you aren't going to get
atmoic line reads. Some systems transfer lines; you're more
likely to have atomicity there.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
One of the drawbacks of being a martyr is that you have to die.

[Malcolm McLean]

"japhy" <ja*******@gmail.comwrote in message

Is there a way to read a line (a series of characters ending in a
newline) from a file (either by descriptor or stream) atomically,
without buffering additional contents of the file?

Not really. The problem is that operating systems are now so complex that it
is difficult to define exactly what we mean by "unbuffered input". For
instance a frequently accessed small file may be memory mapped, completely
transparently to the user.

There are low-level routines provided on most systems that may do what you
want, called "open" insgtead of "fopen" and so forth.
--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

[Richard Heathfield]

bluejack said:

On Mar 2, 2:31 pm, Richard Heathfield <[e-mail]:

>Almost true. Even if we ignore the C99 floating point stuff as being
irrelevant here, we still must not forget sig_atomic_t, which is
defined to be "the (possibly volatile-qualified) integer type of an
object that can be accessed as an atomic entity, even in the presence
of asynchronous interrupts".

Just for my own information, does "asynchronous interrupts" include
thread pre-emption?

Since the Standard doesn't mention threads, it is (alas) impossible to
say within the context of ISO C.

<snip>

mx may be atomic, but the test (mx++ == 0) is not guaranteed to be an
atomic operation?

Correct.

(And I presume there's no way to make such a guarantee in C alone.)

Again, correct.

--
Richard Heathfield
"Usenet is a strange place" - dmr 29/7/1999
http://www.cpax.org.uk
email: rjh at the above domain, - www.