473,396 Members | 1,724 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Any parser can handle 2.1GB+ file?

Well, I tried to use SAX to process a large document. However, we are
hitting an unfortunate limit in the xerces sax parser. I wish they
had used longs or unsigned ints (see below, which is throwing a
runtime exception) to keep track of the position in the document.
Atleast I'm guessing thats whats happening here.

Any ideas?
(from a utils class is xerces)
221 public int addString(int offset, int length) {
222 int chunk = offset >> CHUNK_SHIFT;
223 if (chunk != fChunk) {
224 if (fPreviousChunk == null)
225 throw new RuntimeException(new
ImplementationMessages().createMessage(null,
ImplementationMessages.INT_PCN, 0, null));
226 return fPreviousChunk.addString(offset, length);
227 }
228 int lastChunk = (offset + length - 1) >> CHUNK_SHIFT;
229 if (chunk == lastChunk) {
230 addRef();
231 return fStringPool.addString(this, offset &
CHUNK_MASK, length);
232 }
233 String str = toString(offset & CHUNK_MASK, length);
234 return fStringPool.addString(str);
235 }

here's the java stack trace...

java.lang.RuntimeException: Internal Error: fPreviousChunk == NULL
at org.apache.xerces.framework.XMLParser.parse(Unknow n Source)
at org.apache.xerces.framework.XMLParser.parse(Unknow n Source)
at Test.main(Test.java:177)
Jul 20 '05 #1
0 1518

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: ggg | last post by:
There are over 10 different XML reports I can download from someone else's server. I've made a class to parse the XML into an array. So basically I just have the start_tag() track what the...
13
by: Paulo Pinto | last post by:
Hi, does anyone know of a Python package that is able to load XML like the XML::Simple Perl package does? For those that don't know it, this package maps the XML file to a dictionary.
3
by: gabriel | last post by:
Is there such a parser out there? I'm using XMLSPY.. can its native parser handle imbedded Schematron and validate an instace against it? /Gabriel
1
by: Bruce Pullen | last post by:
db27.2 (fp13) on AIX 5.2. The following error is generated when attempting a reorg or, more precisely, when the file associated with the reorg tablespace reaches 1Gb. SQL2216N SQL error...
25
by: Ali-R | last post by:
Hi, Is there a parser which parses CSV files? Thanks for your help. Reza
6
by: Sakcee | last post by:
html = '<html><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" <head></head> <body bgcolor=#ffffff>\r\n Foo foo , blah blah </body></html>' >>> import htmllib >>> import...
4
by: siddharthkhare | last post by:
Hi All, I need to parse certain text from a paragraph (like 20 lines). I know the exact tags that I am looking for. my approach is to define a xml (config) file that defines what tag I am...
3
by: rony | last post by:
Hi, I am looking for component which allows me to parse my xml file. the reason i am asking this, is because my xml files are huge it can reach as far as 1GB more or less. the time to parse such...
3
by: jeremy.gehring | last post by:
Hey all, OK I'm not much of a PHP programmer; but needs must as they say. I have written AJAX file upload system that uses a PERL CGI script so that a PHP script can get the progress (nifty...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.