Options for real-time plotting of server-side data?

I'm interested in setting up a web page where live data can be
displayed in real-time on the web page.

For example: I would like to display a (nice looking) graph of some
data value versus time and have the graph update every second without
the user having to do anything like hit a refresh button. The data to
plot is readily available from an application running on the server - I
can expose it in whatever way is needed (currently easily available via
XML-RPC).

I've got lots of programming experience but very little web development
experience and am struggling to decipher the many web development
options that are out there. The sheer number of web development options
is a bit bewildering to me right now.

What are my options? Does anyone know of any existing packages that
enable dynamic plotting viewable by a standard web browser?

As far as far as languages are concerned for dynamic web development
I've been investigating what is out there and below is my extremely
short and terrible summary of what seems to be available. This is
definitely NOT a troll - just trying to figure out my options:

PHP
- everybody's using it... seems very good for dynamic web pages, but
possibly only good for dynamically creating static web pages (as
far as the user is concerned). Unsure if the latter is true.
- Seems to be a good graphing package available (JpGraph), but it
only seems to be able to provide static graphs.
- http://www.aditus.nu/jpgraph/index.php
- server side execution

Perl
- similar enough to PHP as far as capabilities... not looking for a
PHP/Perl comparison, just loking for what can do what I need.

JavaScript
- client side execution
- Unsure if there are any graphical capabilities at all here
- Can I somehow embed an obect and use Javascript to change it's
properties?

ActionScript/Flash
- In general it seems that Flash/Macromedia is a logical solution to
real-time updating of graphics
- Unclear at this point how ActionScript connects with Flash et al
- Going Macromedia could be expensive due to $1k per developer for
Studio 8 (not insurmountable but a factor)
- May be some free routes such as MTASC but not entirely sure of the
capability or implications here at this point
- http://www.mtasc.org/

CGI
- dead/old?
- Likely no graphics

ASP
- dead/old?
- heavy/specific server requirements?
- not keen on this route for some reason

Whatever solution I end up with should ideally be small in footprint
and also in usage of server resources. I will have my own server...
likely some Apache flavor or other.

Any suggestions on how to achieve this real-time graphics updating
based on live server-side data is appreciated!

Suggestions on a more appropriate newsgroup or other forum to post this
to are also appreciated.

Thanks!
Russ

Russ <russandheather@gmail.com> wrote:[color=blue]
> For example: I would like to display a (nice looking) graph of some
> data value versus time and have the graph update every second without
> the user having to do anything like hit a refresh button. The data to
> plot is readily available from an application running on the server - I
> can expose it in whatever way is needed (currently easily available via
> XML-RPC).
>
> - everybody's using it... seems very good for dynamic web pages, but
> possibly only good for dynamically creating static web pages (as
> far as the user is concerned). Unsure if the latter is true.
> - Seems to be a good graphing package available (JpGraph), but it
> only seems to be able to provide static graphs.
> - http://www.aditus.nu/jpgraph/index.php
> - server side execution[/color]

I just used jpgraph for the first time last week. Downloaded it during a
meeting and was able to have an application up and running later in the same
meeting - it was very straightforward to use and the documentation has
copious examples.

I'm not sure what you mean about "static graphs" - it will not produce
animated graphs, but you can regenerate them as often as you please and use
refreshes or other methods to keep it updating in client browsers.

miguel
--
Hit The Road! Photos from 38 countries on 5 continents: http://travel.u.nu
Latest photos: Burma; Hong Kong; Macau; Amsterdam; Grand Canyon; Amman

Russ wrote:
[color=blue]
> I'm interested in setting up a web page where live data can be
> displayed in real-time on the web page.
>
> For example: I would like to display a (nice looking) graph of some
> data value versus time and have the graph update every second without
> the user having to do anything like hit a refresh button. The data to
> plot is readily available from an application running on the server - I
> can expose it in whatever way is needed (currently easily available via
> XML-RPC).
>
> [snip]
>
> What are my options? Does anyone know of any existing packages that
> enable dynamic plotting viewable by a standard web browser?
>
> [snip]
>
> PHP
> Perl
> CGI
> ASP[/color]

These are just different languages/environments for server-side programming
(CGI is a protocol to call programs written in any programming languages
from the webserver). Everyone probably has a way to pull data from some
datasource and create a pretty graph of it. But without further support on
the client-side all you can do is to build a self-refreshing page with a
dynamically created image embedded. This can consume pretty much bandwidth
(the whole image has to be transferred on each refresh).

Even with special client-side support, you'll need some server-side
programming to deliver the data to the clients (if you say, you can already
expose it via XML-RPC, then this part is already done).
[color=blue]
> JavaScript[/color]

Using XML-RPC shouldn't be a major problem (keyword AJAX...)
Latest Mozilla/FireFox versions (AFAIK only unreleased CVS versions) and
Safari (I believe...) support a new CANVAS element that can be used to draw
arbitrary graphics using JS. But this is fairly new and experimental...
[color=blue]
> ActionScript/Flash
> - In general it seems that Flash/Macromedia is a logical solution to
> real-time updating of graphics
> - Unclear at this point how ActionScript connects with Flash et al[/color]

ActionScript for Flash is what JavaScript is for HTML. ActionScript has some
XML-RPC support which you could use to pull data from the server.

You could also you a Java applet to draw the graphs and pull data from the
server.

My (very personal) suggestion: I've done enough of Flash/ActionScript
programming to prefer the Java solution, although I haven't done Java for
years... ;)

--
Benjamin Niemann
Email: pink at odahoda dot de
WWW: http://www.odahoda.de/

> I'm not sure what you mean about "static graphs" - it will not produce[color=blue]
> animated graphs, but you can regenerate them as often as you please and use
> refreshes or other methods to keep it updating in client browsers.[/color]

What I mean is that it seems that what this is doing is generating plot
images on the server side and sending them to the client. I'd like to
just be able to update the graph only to avoid sending a whole image...
this is for bandwidth and flicker reasons (plus I may have multiple
graphs updating at different rates). The main thing I guess I'm after
is I'd like to make my web page look like an application and not a
repeatedly refreshing html page.

> These are just different languages/environments for server-side programming[color=blue]
> (CGI is a protocol to call programs written in any programming languages
> from the webserver). Everyone probably has a way to pull data from some
> datasource and create a pretty graph of it. But without further support on
> the client-side all you can do is to build a self-refreshing page with a
> dynamically created image embedded. This can consume pretty much bandwidth
> (the whole image has to be transferred on each refresh).
>
> Even with special client-side support, you'll need some server-side
> programming to deliver the data to the clients (if you say, you can already
> expose it via XML-RPC, then this part is already done).[/color]

I pretty much knew I needed some client side support and have no
problem with that as long as the needed client support is
cross-platform, semi-universal, and non-crippling (eg: JavaScript,
Flash Player). My problem is I don't know what is strictly server side
and what is not? eg: Is PHP only server side or can there be some
client side stuff? From what I've seen so far it looks like it (and
Perl) are server only and that JavaScript is client only.

Thanks for the CGI tidbit... didn't know that.

I really only listed the things I did because I was/am coming from a
pretty clean web development slate.

As for server data availability, this is no problem. I did see that
some of the languages support XML-RPC (or someone has made a tool for)
, but this part is very easy to change to match what seems to be the
tough part to me - the nice web app.
[color=blue][color=green]
> > JavaScript[/color]
> Using XML-RPC shouldn't be a major problem (keyword AJAX...)
> Latest Mozilla/FireFox versions (AFAIK only unreleased CVS versions) and
> Safari (I believe...) support a new CANVAS element that can be used to draw
> arbitrary graphics using JS. But this is fairly new and experimental...[/color]

Since posting I've been looking into Javascript/Ajax (unclear so far on
the distinction - Ajax is a new one to me) options and am quite amazed
what can be done. Some examples of what I've found (some not plotting,
but generally making nice looking applications):

www.bindows.net
- Interesting claims about it all being server side?!
- seems to only work nicely with IE...
- They have decent looking example of plotting and other surprising
widgets:
- http://www.bindows.net/bindows/samples/inline/

qooxdoo.oss.schlund.de
- More JS capabilities to make full applications
- no plot I can find, but there is a (lame) demo of the canvas you
mentioned:
- qooxdoo.oss.schlund.de/demo/release/public/test/user/Canvas_5.html
- other demos available in the top-right combo o fthat last link
are impressive as far as making something look like an application,
but it seems quite slow overall.

http://www.walterzorn.com/dragdrop/dragdrop_e.htm
- General JS-only capability I didn't think was possible
- no plots

I'm still digging about for stuff.
[color=blue]
> ActionScript for Flash is what JavaScript is for HTML. ActionScript has some
> XML-RPC support which you could use to pull data from the server.[/color]

So far I'm very impressed with the potential for Flash. I now somewhat
get how it is organized with the combination of MXML and ActionScript
working together to generate SWF's. If I'm right your statement shoudl
really be "ActionScript is for MXML is what JavaScript is for HTML" -
plus you need the Flash player on the client to use either MXML or AS.
Right? It seems that it can do everything I want it to so far,
including having built in graphing controls (or widgets or whatver
they'd be called in Flash). eg: I found an example of a built in line
plot (and others):
livedocs.macromedia.com/flex/15/flex_docs_en/wwhelp/wwhimpl/js/html/wwhelp.htm

I located a document that describes the Flash platform quite well at
http://www.macromedia.com/platform/w...m_overview.pdf and on a
scan through it has me pretty excited. I'll be reading it over
tonight.
[color=blue]
> You could also you a Java applet to draw the graphs and pull data from the
> server.[/color]

I've not much Java experience, even as a user, but from what I have
seen I've always thought that Java apps were slow to load and were very
obvious that they were Java apps. I never liked the Java engine
lurking in my system tray either, but that is obviously not a real
concern. The Java client-side applications is big, too... no? If
you've got a link to a good Java App (preferrably a real time updating
one) I'd love to see it in order to geta feel for the potential, and
hopefully
[color=blue]
> My (very personal) suggestion: I've done enough of Flash/ActionScript
> programming to prefer the Java solution, although I haven't done Java for
> years... ;)[/color]

Personal suggestions/ideas are what I'm after! I'm glad to have found
this general forum... posting to a JS or Flash or PHP or whatever forum
would likely have been a disaster.

Without too much detail (and to avoid a war) is there any one
particular thing that makes you prefer using Java over Flash?
Development environment? General comfort? Overall capability?

Thanks for the responses,
Russ

Russ wrote:[color=blue]
> ......................
> What are my options? Does anyone know of any existing packages that
> enable dynamic plotting viewable by a standard web browser?
> .....................
> As far as far as languages are concerned for dynamic web development
> I've been investigating what is out there and below is my extremely
> short and terrible summary of what seems to be available. This is
> definitely NOT a troll - just trying to figure out my options:
> ....................
> Perl
> - similar enough to PHP as far as capabilities... not looking for a
> PHP/Perl comparison, just loking for what can do what I need.
> ..................[/color]
GD::Graph module. Get it from cpan.org. (If you're on shared server
you can install it into your own CGI directory if it is not already
running on the server. It probably won't be, but check.)
Does bar, line, and pie charts. Probably more by now, but I haven't
checked its capabilities in a while. Quite easy to program.
Display it with CGI using the CGI module.[color=blue]
> .....................
> CGI
> - dead/old?[/color]
Dead, absolutely not. Old, yes, and that's no problem. The faster mod
Pearl is available on some but not all servers. I have found regular
CGI to be more than fast enough for most uses.[color=blue]
> - Likely no graphics[/color]
You create .jpg, .gif, .png, etc. images with your graphing program.
CGI can generate normal web pages that display these. A number of
languages and servers support the CGI interface. Perl and Python for
starters.

--
mbstevens
http://www.mbstevens.com

mbstevens wrote:
......................[color=blue][color=green]
>> CGI
>> - dead/old?[/color]
>
> Dead, absolutely not. Old, yes, and that's no problem. The faster mod
> Pearl is available on some but not all servers. I have found regular
> CGI to be more than fast enough for most uses.[/color]

I may have been a bit confusing there. Mod Perl is a version of Perl
that keeps running on the server without having to create a new program
instance each time a script is called from a web page. The Perl is
faster, not the CGI. But regular Perl and CGI is quite fast enough for
most uses.

BTW, I don't think you should be worrying too much about refreshing the
whole page. If it's programmed and marked up right, there won't be more
than a flicker.

"Russ" <russandheather@gmail.com> wrote in
news:1129750568.959509.212240@o13g2000cwo.googlegr oups.com:
[color=blue]
> I'm interested in setting up a web page where live data can be
> displayed in real-time on the web page.[/color]

<Snipped a bunch>

Having programming experience/knowledge as you do, you're at a bit of a
disadvatage for web develpment. You'll have to learn to forget the
"program loop" way of thinking and learn how to create "a page." You
have an advantage in the programming knowledge though when it comes to
coding and logic. Learn how to think in linear terms towards a final
output.
[color=blue]
> PHP
> - everybody's using it... seems very good for dynamic web pages, but
> possibly only good for dynamically creating static web pages (as
> far as the user is concerned). Unsure if the latter is true.
> - Seems to be a good graphing package available (JpGraph), but it
> only seems to be able to provide static graphs.
> - http://www.aditus.nu/jpgraph/index.php
> - server side execution[/color]

With the above in mind, I'd recommend PHP as it is widely supported,
there are also news groups like this one for discussing PHP issues
(same is true of most other languages too). And I'm finding it very
easy to use because of it's integration into HTML. It's very strange
at first swapping between two "languages" in the same document. Since
you are also just beginning with HTML, I think learning to put HTML,
CSS, and PHP all together from the beginning makes a lot of sense.

See:
http://www.w3.org/TR/html4/
http://www.w3.org/Style/CSS/
http://www.php.net/
[color=blue]
> CGI
> - dead/old?
> - Likely no graphics[/color]

That's the protocol used to "launch" server based programs.

--
Stan McCann "Uncle Pirate" http://stanmccann.us/pirate.html
Webmaster/Computer Center Manager, NMSU at Alamogordo
http://alamo.nmsu.edu/ There are 10 kinds of people.
Those that understand binary and those that don't.

Russ <russandheather@gmail.com> wrote:[color=blue]
> Without too much detail (and to avoid a war) is there any one
> particular thing that makes you prefer using Java over Flash?
> Development environment? General comfort? Overall capability?[/color]

Java has a much larger development community and more tools, documentation,
and examples available.

Flash will run much more smoothly in the browser, is less likely to have
compatibility issues, and the graphs will look 50 times better with the same
amount of effort.

You can generate Flash from PHP and other languages.

miguel
--
Hit The Road! Photos from 38 countries on 5 continents: http://travel.u.nu
Latest photos: Burma; Hong Kong; Macau; Amsterdam; Grand Canyon; Amman

Russ wrote:[color=blue]
> I'm interested in setting up a web page where live data can be
> displayed in real-time on the web page.
>
> For example: I would like to display a (nice looking) graph of some
> data value versus time and have the graph update every second without
> the user having to do anything like hit a refresh button. The data to
> plot is readily available from an application running on the server -
> I can expose it in whatever way is needed (currently easily available
> via XML-RPC).[/color]

I strongly suggest you take a look at PHP/SWF Charts:
http://www.maani.us/charts/index.php

They promise "Live and interactive chart updates without reloading the web
page ".

I tried the software earlier this year and it looked ok and worked well. I
have not tested the live data part but I guess it's just what you're looking
for.

Personally I prefer GlobFX's Swiff Chart Generator
(http://www.globfx.com/products/swfchartgen/), it feels a lot more finalized
and pack lots of features. No live data yet, though.

HTH

--
Suni

> I strongly suggest you take a look at PHP/SWF Charts:[color=blue]
> http://www.maani.us/charts/index.php[/color]

Thanks for the great link! This is very promising.
[color=blue]
> They promise "Live and interactive chart updates without reloading the web
> page ".[/color]

Their implementation of this is documented at:
http://www.maani.us/charts/index.php...nu=live_update
Perusing this it seems that the way it works is that the flash
app/chart does a scheduled run of a server-side PHP script which can
then re-generate a new graph and send it back to the client to update
the display. I think I have that right... I wonder if it can be
incremental?

Does anyone have any experience with these Flash charts? I wonder how
it would fare if doing a line plot of a few thousand data points...

Or what about whether swf files, once generated, have an interface so
that you can change the contents on the fly?

Again - awesome link.
[color=blue]
> Personally I prefer GlobFX's Swiff Chart Generator
> (http://www.globfx.com/products/swfchartgen/), it feels a lot more finalized
> and pack lots of features. No live data yet, though.[/color]

Also interesting but as you say - no live data.

I find it interesting that both of these solutions seem to be
generating swf files at the server (using the Macromedia provided chart
objects, no less!) and don't appear to need anything specifically from
Macromedia at the server.

Taking a peek at PHP/SWF Charts' implementation at the server it
appears to be done by having an swf file already available on the
server for every chart type that exists. And there is not a heck of a
lot of PHP code. This is very interesting and may answer my previous
question as to whether swf files are tweakable after generation... hmm.

You guys are a great help - thanks! Keep it coming... :)

Russ

Russ <russandheather@gmail.com> wrote:[color=blue]
> I find it interesting that both of these solutions seem to be
> generating swf files at the server (using the Macromedia provided chart
> objects, no less!) and don't appear to need anything specifically from
> Macromedia at the server.[/color]

You don't need anything rom Macromedia to generate SWF files. For example,
see here:

http://my2.php.net/ming

miguel
--
Hit The Road! Photos from 38 countries on 5 continents: http://travel.u.nu
Latest photos: Burma; Hong Kong; Macau; Amsterdam; Grand Canyon; Amman

Russ wrote:[color=blue]
> I'm interested in setting up a web page where live data can be
> displayed in real-time on the web page.
>[/color]
Depends a bit on your target audience. I'm on a slow dial up and don't
have flash loaded. Also my proxy and firewall are very restrictive about
what javascript is allowed to do. (mostly it's turned off). So for
maximum coverage you need to stay with server side scripts.

Unless your graphs contain information that your audience will want to
see updated every few second, I would re think the approach.