Memory Consumption in JDOM

Vijay Anandh

Hi All,
I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??
Also if there are any resources on jdom benchmarking please let me
know the links.
( BTW, I am using the latest JDOM build ).

Thanks in Advance for your help
Vijay Anandh

May 13 '06 #1

Subscribe Reply

2128

Vijay Anandh

JDOM experts........ , help me please on my above question.
I am eagerly waiting...,

May 13 '06 #2

Jürgen Kahrs

Vijay Anandh wrote:

I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??

No, probably not. In this newsgroup, we have
explained many many times what a DOM does:
Read the complete XML file into memory.
This procedure needs at least so much memory
(RAM) as the XML file takes on disk.

So, unless you have many GB of RAM, the answer is no.
Use Google to find all the other details that
have been posted in this newsgroup earlier.

May 13 '06 #3

Joe Kesselman

Jürgen Kahrs wrote:

So, unless you have many GB of RAM, the answer is no.

Slight quibble: Most systems these days support virtual memory, so it is
theoretically possible to build models that exceed the size of physical
memory if you're willing to spill to disk. But that costs performance
and Java itself may impose some memory-size limits.

There are two possible approaches. One is to do active memory
management. Another is to replace the JDOM (which is a rather naive
design, I believe, consuming an object per node) or DOM with a more
compact data model; this is part of why we invented the DTM model for Xalan.

If you don't want to do a lot of coding to reinvent those solutions, you
may want to investigate XML databases. Managing huge amounts of data is
*supposed* to be what they're tuned for, and they should have applied
these sorts of tricks. (Note that I say "should"; I can't advise you on
which ones are good or not other than to say that I'm favorably
impressed with IBM's new XML support in the latest version of DB2. But
I'm an IBMer, so I may be biased.)
--
() ASCII Ribbon Campaign | Joe Kesselman
/\ Stamp out HTML e-mail! | System architexture and kinetic poetry

May 13 '06 #4

Vijay Anandh

Thanks for your replies..
I have tried benchmarking my JDOM application
on our solaris 8 server( more RAM now ).

sizes of files to create 10M 20M 30M 40M
increase heap size to none none 128M 128M
~ time taken in seconds 2 4 7 11
And fortunately the maximum file size I will be creating is ~70M.
So I will right now persist with JDOM ( while increasing my JVM heap
size )
and test its limits and then plan for XML databases

Vijay Anandh

May 13 '06 #5

Similar topics

1720

Object member functions and runtime memory consumption

by: He Shiming | last post by:

Hi, I have a question regarding memory consumption of class/object member functions. Say that I have: class A { char szAbc; char* getString(); }

C / C++

2251

ASP.NET WP memory consumption problem & debugging on AppDomain bas

by: Teemu Keiski | last post by:

Hi, I have following type of scenario (also explained here http://blogs.aspadvice.com/joteke/archive/2005/01/10/2196.aspx ) We have problematic web server (wink2 Standard, 1.5GB of physical memory, SQL 2000 in same box, framework 1.0 and 1.1 installed, apps use mainly 1.0) whose aspnet_wp.exe's memory consumption increases slowly but surely, leading to process restart eventually and then again recollecting memory etc etc

ASP.NET

1399

Weird memory consumption problem.

by: Bo Peng | last post by:

Dear list, I spent the last 12 hours in catching this bug (?) and what I found out is very difficult to explain: Basically, I need to call a user-provided function many times, with a tuple as parameter. C/C++ side: (class A, constructed using a python function m_func)

Python

1721

multythreading app memory consumption

by: Roman Petrichev | last post by:

Hi folks. I've just faced with very nasty memory consumption problem. I have a multythreaded app with 150 threads which use the only and the same function - through urllib2 it just gets the web page's html code and assigns it to local variable. On the next turn the variable is overritten with another page's code. At every moment the summary of values of the variables containig code is not more than 15Mb (I've just invented a tricky way to...

Python

2057

Memory consumption of generic lists

by: Hannes | last post by:

Hi guys! I use a generic list which, once filled, will not change. It contains a few 100'000 entries, so I'd like to limit its memory consumption. If I set its capacity to its count after filling it, does the extra unused memory (due to its probably high capacity by this time) get freed up do I have to copy it into a new list where I set the capacity before filling? It is not possible to know how many entries it will contain before...

C# / C Sharp

9221

Reducing memory consumption

by: Bruno Barberi Gnecco | last post by:

I'm using PHP to run a CLI application. It's a script run by cron that parses some HTML files (with DOM XML), and I ended up using PHP to integrate with the rest of the code that already runs the website. The problem is: it's eating more memory than a black hole. It eats the current limit of 256MB set in php.ini, in an application that would hardly consume 4MB if written in C. I don't care if this application takes much longer to run...

PHP

1925

How can I reduce the memory consumption?

by: liubin | last post by:

I have created a simple program using C#/.NET. It is very small program without explicit system resources request in the code. However, the memory consumption is even above 17M(shown in Windows Task Manager), much higher than 5M of the same one developed with MFC before. Now I wonder if there is any effective way to optimize .Net program achieving following two objectives: 1. Reduce system resources consumption of the program created...

C# / C Sharp

1318

Minimize the physical and virtual memory consumption

by: dattaforit | last post by:

Hello All, I have a Window service developed using Visual Studio .Net 2003. I am using VC++ .Net for the service development. The physical and virtual memory consumption is very high. How should i minimize this memory consumption. Please give me some clue to achieve this. Thanks

.NET Framework

1767

memory consumption!

by: =?Utf-8?B?U2FuZGVlcCBUaG9wcGls?= | last post by:

Hi We built a Windows application on .Net 2 framework using Infragistics controls. We got a bug reported from our client complaining high physical memory consumption showing in the task manager and the memory suddenly drops off as they minimize the application. As we analysed the problem we could find the application seems to consume 55000K and drops off to 3000K as I minimize the application, but the strange thing I noiticed is that...

.NET Framework

2277

limiting memory consumption of Python itself or a dict in a Python program

by: Jonas Maurus | last post by:

Hello everybody, I'm pondering the following problem: I want to write a Python program that receives messages via SMTP and stores them in a dict or an array. For my purposes it would be important that all received mail would be kept in RAM and not cached out to disk. If a new message comes in that can't fit in the allocated memory, a number of old messages would be discarded.

Python

7962

What is ONU?

by: marktang | last post by:

ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...

General

7884

Changing the language in Windows 10

by: Hystou | last post by:

Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...

Windows Server

8267

Problem With Comparison Operator <=> in G++

by: Oralloy | last post by:

Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...

C / C++

8380

Maximizing Business Potential: The Nexus of Website Design and Digital Marketing

by: jinu1996 | last post by:

In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...

Online Marketing

8024

The easy way to turn off automatic updates for Windows 10/11

by: Hystou | last post by:

Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...

Windows Server

8258

Discussion: How does Zigbee compare with other wireless protocols in smart home applications?

by: tracyyun | last post by:

Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...

General

3921

Windows Forms - .Net 8.0

by: adsilva | last post by:

A Windows Forms form does not have the event Unload, like VB6. What one acts like?

Visual Basic .NET

1493

How to add payments to a PHP MySQL app.

by: muto222 | last post by:

How can i add a mobile payment intergratation into php mysql website.

PHP

1229

Comprehensive Guide to Website Development in Toronto: Expert Insights from BSMN Consultancy

by: bsmnconsultancy | last post by:

In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

General