473,857 Members | 2,173 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

File objects? - under the hood question


I didn't come across any illuminating discussion via Google, thus my question here (though it may be a neophyte question.) I am interested in the workings under the hood of Python's access of "files".

What is actually happening at the various stages when I create a file object and "read" it?

(1) >>> f = file("C:/GuidosParrot.tx t","r")

(2) >>> hesjustsleeping = f.read()

At (1) have I loaded the file from hard drive into RAM when I create the file object? What does this object know and how did it find it out?

At (2) am I loading the file contents into RAM, or just assigning what is already in RAM to a variable?

Where is the work split between the OS and Python? I assume the OS is responsible for "presenting " the file to Python, so perhaps the OS assembles this file from the blocks on disk and loads it into RAM at the time the file object is created? Or would the OS simply have pointers that can assemble the file, and pass those pointers to Python?

Perhaps I've answered my question and the under-the-hood mechanics are handled on the OS side, and Python is just making requests of the OS...
My brain-teaser: What I'd like to do is read the last ~2K of a large number of large files on arbitrary servers across the net, without having to read each file from the beginning (which would be slow and resource inefficient)...


Eric Pederson
http://www.songzilla.blogspot.com
::::::::::::::: ::::::::::::::: :::::
domainNot="@som ething.com"
domainIs=domain Not.replace("s" ,"z")
ePrefix="".join ([chr(ord(x)+1) for x in "do"])
mailMeAt=ePrefi x+domainIs
::::::::::::::: ::::::::::::::: :::

Jul 18 '05 #1
3 1323
On Tue, 18 Jan 2005 22:53:10 -0800, Eric Pederson wrote:
Perhaps I've answered my question and the under-the-hood mechanics are
handled on the OS side, and Python is just making requests of the OS...
Almost by definition, the only correct way to read a file is to use the
file system, which on current operating systems means going the the OS
kernel.
My brain-teaser: What I'd like to do is read the last ~2K of a large
number of large files on arbitrary servers across the net, without having
to read each file from the beginning (which would be slow and resource
inefficient)...


"Across the net" is not specific enough. There are generally ways to do
that, subject to the support of the various relevant servers, but what
protocol are you talking about using? HTTP? FTP? NFS? Generally you issue
some sort of command to get the length, then some sort of "seek" or
"continuati on" command to get to the end, but the details are different
for each protocol. (Unless someone has a convenient flattening library
around? I have something almost like that in my personal collection except
I have no intention of supporting "seek" behavior for various technical
reasons.)

And note that with the possible exception of that last one, there is no
relationship between these two questions. (Maybe you know that, maybe you
don't. :-) ) There is no argument you can pass to file() that will read an
HTTP file. (Pedants may note this isn't an absolute truth but it's "true
enough" that it's not worth sweating the details if you're still working
out what "file()" does.)
Jul 18 '05 #2
Jeremy responds:
[kind enough not to mention I must have had only 10% of my brain cells functioning when I posted]
And note that with the possible exception of that last one, there is no
relationship between these two questions.

Right, I just want there to be.

There is no argument you can pass to file() that will read
an
HTTP file.

A file is a file, no matter where it resides; yes I know it's not that simple.

Here the sort of thing (seek, then read) I think I want:
IDV2=open(("htt p://musicsite.com/song453.mp3","r b")[:-128]) song453.tags=ID V2.read() len(song453.tag s)


128
But it's not a Python problem. :-(
Thanks for the responses and indulgence.
I'm OK now - the repair man fixed the coffee pot.

Eric Pederson
http://www.songzilla.blogspot.com

Jul 18 '05 #3
On Thu, 20 Jan 2005 21:06:31 -0800, Eric Pederson wrote:
Here the sort of thing (seek, then read) I think I want:
IDV2=open(("htt p://musicsite.com/song453.mp3","r b")[:-128]) song453.tags=ID V2.read() len(song453.tag s)
128
But it's not a Python problem. :-(


OK, HTTP. It's true that it isn't a Python problem, but the fact that this
is possible and even easy isn't generally known, since it involves
actually understanding HTTP as more than a protocol that says "give me
this file". :-{

You need to use the Range header in HTTP to request just the end. urllib
doesn't seem to like the Range header (it interprets the 206 response that
results as an error, at least in 2.3.4 which I'm using here, which I would
consider a bug; 2xx responses are "success"), but you can still do it with
httplib:

Python 2.3.4 (#1, Oct 26 2004, 20:13:42)
[GCC 3.4.2 (Gentoo Linux 3.4.2-r2, ssp-3.4.1-1, pie-8.7.6.5)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
import httplib
connection = httplib.HTTPCon nection("www.je rf.org")
connection.requ est("GET", "/", headers = {"Range": "bytes=-100"})
response = connection.getr esponse()
response.read()

'rch Google -->\r\n\t\t\t\t\t </DIV>\r\n\t\t\t\ t\t<P CLASS="Seperato r" />&nbsp;</P>\r\n\t\t\t<di v>\r\n\t\t</body>\r\n\t</html>\r\n'

The bad news is I think you would have to chase redirects and such on your
own. Hopefully a urllib expert will pop in and show how to quickly tell
urllib to chill out when it gets a 206; I'm pretty sure it's easy, but I
can't quite rattle off how. Or maybe 2.4 has a better urllib.
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

12
2605
by: Woodster | last post by:
I currently have some code for an application that is running on Win32. I have tried to keep anything not directly gui related as separate as possible for portability reasons, including file access. Now has come the time to try and implement the program on a Windows CE platform. Problem is, that the MFC CArchive class uses an integer file pointer and calls to open, close etc and my file handling class is doing everything with FILE *, such...
25
4155
by: cppaddict | last post by:
I'd like to know what goes on under the hood when methods return objects. Eg, I have a simple Point class with two members _x and _y. It's constructor, copy constructor, assignment operator and additon operator (which returns another Point object, and which my question is about) are as follows: Point::Point(int x, int y) : _x(x), _y(y) { }
9
2533
by: cppaddict | last post by:
I have a method that uses a fairly large object. The choice is between having a local object in the method or a static member object that the method uses. ------CHOICE 1--------- int MyClass::myMethod(int param) { MyBigObject o(); //initialize o based on param //do calculations with o
11
4328
by: Dale | last post by:
How to recognize whether file has XML format or not? Here is the code segment: XmlDocument* pDomDocument = new XmlDocument(); try { pDomDocument->Load(strFileName ) ; } catch(Exception* e) {
106
6522
by: xtra | last post by:
Hi Folk I have about 1000 procedures in my project. Many, many of them are along the lines of function myfuntion () as boolean on error goto er '- Dim Dbs as dao.database Dim Rst as dao.recordset
161
7948
by: KraftDiner | last post by:
I was under the assumption that everything in python was a refrence... so if I code this: lst = for i in lst: if i==2: i = 4 print lst I though the contents of lst would be modified.. (After reading that
9
13014
by: Tom the Canuck | last post by:
A while back I was playing with C++ and made a simple program with a WAV file as a resource. It worked well and was easy to make. I then went on to try this with VB. I had problems. Can this be done in VB? I have VB6 (Visual Studio 6 Enterprise edition).. Any pointers to get me going in the right direction would help. I like to learn on my own. Making a mistake and figuring out where the error was made enhances the learning experience. It...
1
4129
by: swayze | last post by:
Hi there, We're using php 4.1 and it doesn't seem to have built in support for this. Coming from a dotnet background this surprised me...Anyways, thats a different topic altogether... I'm looking for a php class that can allow me to save a complex associative array as an xml file on our server and also be able to read in back into a complex array when needed. (you'll find an example of the array below). Because of the complexity if...
55
4005
by: tonytech08 | last post by:
How valuable is it that class objects behave like built-in types? I appears that the whole "constructor doesn't return a value because they are called by the compiler" thing is to enable built-in-like behavior for objects of class type. That leads to the "necessity" for the exception machinery so that errors from constructors can be handled. Is all that complexity worth it just to get built-in-like behavior from class objects? Maybe a...
0
9923
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
11082
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10711
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10807
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10394
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7939
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5967
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4190
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.