473,396 Members | 2,030 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

marshal locks program even in thread

Greetings,

It seems that marshal.load will lock the problem if the file object
(in this case a pipe) is not ready to be read from - even if it's done
in a thread.

The use case here is in writing a scripting interface for perforce
using the -G option
(http://www.perforce.com/perforce/doc....html#1040647).
We are using the sub process module to communicate through pipes to
avoid using the hard-drive. Furthermore, because the machine is using
windows we need threads for each pipe (stdin, stdout, stderr)
initialized from the beginning to keep from buffer lock.

If marshal.load would accept a cStringIO object, I would have a
workaround - marshal.load(cStringIO.cStringIO(pipe.read())) because
cStringIO does not have the same problem and I'm not as worried about
the extra memory as I am the hard-drive use.

I can't read the C code yet, but if I had to make a guess I would say
python is grabbing the GIL and then attempting to read from the pipe,
instead of the other way around.

Could someone help me to understand what is going on and suggest a
workaround? I hope I've been clear about the nature of the problem.
Thanks,

--
Zachary Burns
Aim - Zac256FL
Production Engineer
Zindagi Games
Nov 8 '08 #1
0 951

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: kanji | last post by:
Hi ALL, I have written a multithreaded python program where each thread calls a C function (via Python/C extension module) to execute some tasks on a remote node. The number of threads == the...
14
by: Ajay | last post by:
hi! what would happen if i try to access a variable locked by another thread? i am not trying to obtain a lock on it, just trying to access it. cheers
4
by: Jon Davis | last post by:
I have a Windows Forms application where I normally had two threads running, with the second thread started by a main window. The second thread ran in the background just fine. Then I made some...
2
by: Jürgen Devlieghere | last post by:
Hi, We are creating event-driven multi-threaded applications on a daily basis. To help us solving deadlocks, we implemented a CriticalSection class that does dead-lock detection: an attempt to...
14
by: Notre Poubelle | last post by:
Hello, I have a worker thread that needs to call a method that only works on the main/UI thread. One way to accomplish this in managed code is to use Control.Invoke. However, in my case I...
21
by: Mike | last post by:
Hi, The example below shows that result of a marshaled data structure is nothing but a string >>> data = {2:'two', 3:'three'} >>> import marshal >>> bytes = marshal.dumps(data) >>>...
1
by: arun | last post by:
Hello group, I have a static method but this function does not make use of locks. Then what happens if two threads access the local variables and parameters values at different times. What...
2
by: chsalvia | last post by:
The design philosophy behind the boost Lock concept seems to be to implement short-lived, tightly-scoped locks which automatically unlock the mutex when they go out of scope. This design is nice...
1
by: =?Utf-8?B?Um9nZWxpbw==?= | last post by:
hey, I run long queries. like once that take my program about 3 minutes straight to complete.. like I grab over 100,000 rows, and loop throguh each row to update a certain value. etc.. the...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.