Hi list.
I have 2 questions about the Pythonicity of some algorithms. One
question is about generator usage, the other is about producer and
consumer threads
GENERATOR QUESTION
Usually for generators, the usage is like this:
for var in generator():
do_something(var)
I sometimes have cases where I want to use a generator, but I don't
want to stall the main program if the generator function doesn't have
any new data for me (maybe it's retrieving data from a device, but I
want to do other things with the same device if there isn't any data
yet).
So currently, some of my code looks something like this:
def get_serial_data():
while True:
block = request_a_packet_from_serial_device() # Get some raw data
parsed_list = parse_block(block) # Extract a list of records
from the packet
if parsed_list:
# The record contains 1 or more records
for record in parse_list:
yield record
else:
# Sometimes there aren't any records in the packet
yield None
def thread_function_for_handling_serial_device():
record_iter = iter(get_serial_data())
while True:
record = record_iter.next()
if record:
handle_record() # eg: add to a queue for the main thread to process
other_serial_logic1()
other_serial_logic2()
other_serial_logic3()
This works for me. But I'd like to know if this is considered
Pythonic, and if there are other, better ways of doing the above in
Python.
PRODUCER AND CONSUMER THREAD QUESTION
In some of my Python apps I use the producer/consumer model, where the
producer and consumer threads communicate through Queues.
Usually code (in the consumer thread function) looks something like this:
def consumer_thread():
while True:
record = some_queue.get()
process_record(record)
In some cases I'd like to do something unusual to threads consuming a
queue - interrupt them, make them terminate, etc. The problem is that
a lot of the time it's locking, waiting for a record from
some_queue.get()
A few things I've considered:
1) Add a timeout to some_queue.get(), so the loop has a chance to do
other things while waiting for new records. This is problematic
because most of the time there isn't other things for the thread to
do. Also, it won't be able to respond immediately to those events.
2) Add special values to the queue, the purpose of which is to notify
the consumers. This is problematic when you have multiple consumers.
You have to add the exact correct number of 'interrupt' objects to the
queue.
3) Make a custom thread-safe queue class, with an 'interrupt_get'
method. When your app calls queue.interrupt_get(), all threads
currently locking on a queue.get() will continue, but with a
GetInterrupted exception thrown.
Perhaps there is some other Python threading idiom I should be using
in this case. Any suggestions?
David. 2 910
This works for me. But I'd like to know if this is considered
Pythonic, and if there are other, better ways of doing the above in
Python.
From the Python point of view, it's fine. However, it uses busy-wait,
which I consider bad style (in any language).
3) Make a custom thread-safe queue class, with an 'interrupt_get'
method. When your app calls queue.interrupt_get(), all threads
currently locking on a queue.get() will continue, but with a
GetInterrupted exception thrown.
That's what I would do. I'd use the existing Queue class as a base
class, though, rather than starting from scratch.
Regards,
Martin
On May 10, 5:21*am, "Martin v. Löwis" <mar...@v.loewis.dewrote:
This works for me. But I'd like to know if this is considered
Pythonic, and if there are other, better ways of doing the above in
Python.
From the Python point of view, it's fine. However, it uses busy-wait,
which I consider bad style (in any language).
3) Make a custom thread-safe queue class, with an 'interrupt_get'
method. When your app calls queue.interrupt_get(), all threads
currently locking on a queue.get() will continue, but with a
GetInterrupted exception thrown.
That's what I would do. I'd use the existing Queue class as a base
class, though, rather than starting from scratch.
Regards,
Martin
Did you follow the links to Dev-C++? It rules house. I have
drizzle.c compiling in Windows GDI and SDL. I'm poised to double-time
Dev-C++ and Windows GDI, if double-tasking is on the market. Credit
up... what's standard? This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: PHPkemon |
last post by:
Hi there,
A few weeks ago I made a post and got an answer which seemed very logical.
Here's part of the post:
PHPkemon wrote:
> I think I've figured out how to do the main things like...
|
by: van |
last post by:
Writing classes of some algorithms, some parameters needs to be passed into
the classes, the actual algorithms are contained in member functions of
classes.
What I am thinking is having some...
|
by: Leandro Lameiro |
last post by:
Hi folks
Recently I have been discussing with a friend about python ease of
use, and it is really good at this. This friend needed to calculate
the MD5 hash of some files and was telling me...
|
by: Michael B. |
last post by:
I was just thinking about this, specifically wondering if there's any
features that the C specification currently lacks, and which may be
included in some future standardization.
Of course, I...
|
by: Anders Borum |
last post by:
Hello!
I'm starting to dive into algorithms in general (sorting, trees etc.) and am
currently reading a book from Robert Sedgewick called "Algorithms in C++,
3rd. Edition" and would like other...
|
by: vlsidesign |
last post by:
I am a newbie and going through "The C programming language" by
Kernighan & Richie on my own time (I'm not a programmer but I want to
learn because it can save me time in my normal job, and it is...
|
by: arnuld |
last post by:
i am looking for "algorithms in C++" book. Knuth is FULL of
Mathematics, not my kind of author. i checked ACCU and got these
(listing only those that are available in my country:
1. Algorithms...
|
by: =?Utf-8?B?Q3JtTmV3Ymll?= |
last post by:
Hi,
1) I want to hone my problem solving skills and be good at logic. How do I
achieve
this?
2) Where can I find c# puzzles or c# algorithms.
3) How do I print the values of a N X N matrix...
|
by: subramanian100in |
last post by:
Below is my understanding about count algorithms.
Return type of count and count_if algorithms is
iterator_traits<InputIterator>::difference_type.
If the container contains more than...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
| |