473,396 Members | 2,011 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

what is the capapcity fo mmap?

151 100+
I am working on file indexing and found one solution to store keyword index into memory map by converting keyword into integers.But still i am not sure how many digits i can store by using it.
Like i have one keyword "Network Analysis by Graph theory and statistics"
if i convert it into integer it will be too lengthy to store as keyword.
map = (int*)mmap(0, FILESIZE, PROT_READ, MAP_SHARED, fd, 0);
How can i overcome my pblm.

Thanks in advance.

Regards
Manish
Sep 14 '09 #1
1 1781
weaknessforcats
9,208 Expert Mod 8TB
Do you know about association tables?

These tables associate a key with an index value:

Network Analysis by Graph theory and statistics 1
String Theory for Dummies 2
My Life as an Analyst 3

etc...

The table usually has a value at the beginning of the table for the next index value. One alternative is to have the next index value be the offset into the database. If you do this, your database can be on disc where it can be far larger than your computer memory and never needs to be loaded into memory. Now you can have several computers accessing the database at once. A further variation on this is to identify a database file name with that offset. Now you can look up the integer representing the database and then access at the offset in the association table. Now your database can be on multiple disc drives while being accessed by multiple computers. Etc...

So you don't convert a key to an integer, instead you associate to an integer.

The association table can be written to disc and then reloaded the next time you want to run the program.

You can also provide reverse assocation by associating an integer value to its key.

You can provide fopr multiple keys by associating several keys to the same integer value.

The association table is usually stored as a binary tree.
Sep 14 '09 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

26
by: Chris Lasher | last post by:
Hello, I have a rather large (100+ MB) FASTA file from which I need to access records in a random order. The FASTA format is a standard format for storing molecular biological sequences. Each...
3
by: subnet | last post by:
I'm trying to write a very simple program that uses a signal-based synchronization between two processes. Here's it: ----------------------------------------------- /* The world's simplest...
1
by: James T. Dennis | last post by:
I've been thinking about the Python mmap module quite a bit during the last couple of days. Sadly most of it has just been thinking ... and reading pages from Google searches ... and very little...
1
by: koara | last post by:
Hello all, i am using the mmap module (python2.4) to access contents of a file. My question regards the relative performance of mmap.seek() vs mmap.tell(). I have a generator that returns...
2
by: Neal Becker | last post by:
On linux, I don't understand why: f = open ('/dev/eos', 'rw') m = mmap.mmap(f.fileno(), 1000000, prot=mmap.PROT_READ|mmap.PROT_WRITE, flags=mmap.MAP_SHARED) gives 'permission denied', but...
0
by: Kris Kennaway | last post by:
If I do the following: def mmap_search(f, string): fh = file(f) mm = mmap.mmap(fh.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) return mm.find(string) def mmap_is_in(f, string): fh =...
0
by: Gabriel Genellina | last post by:
En Thu, 29 May 2008 19:17:05 -0300, Kris Kennaway <kris@FreeBSD.org> escribió: Looks like you should define the sq_contains member in mmap_as_sequence, and the type should have the...
0
by: Akira Kitada | last post by:
Hi Marc-Andre, Thanks for the suggestion. I opened a ticket for this issue: http://bugs.python.org/issue4204 Now I understand the state of the multiprocessing module, but it's too bad to see...
0
by: M.-A. Lemburg | last post by:
On 2008-10-25 20:19, Akira Kitada wrote: Thanks. The errors you are getting appear to be related to either some missing header files or a missing symbol definition to enable these - looking...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.