Miva Script Database
Information Officer, NISS
L. Databases
Databases are where Miva really shines, and there is far too much to introduce here beyond the basics. There will also be no mention of accessing ODBC data sources since the author's experience is only with the XBASE databases that Miva handles under Unix. The XBASE/DBASE IV (.dbf) format that Miva supports has its benefits and its downsides:
Pro:
- very standard, most programs will export into DBase, including Access, which means you can get your data onto the web very quickly indeed
- very fast at find and relatively quick to index
- supports most basic fields types, including the 'unlimited' text field called MEMO
Con:
- outdated with respect to the database formats available nowadays, the limitations on field length, index complexity and data types won't suit all developers
- not truly relational, although relationships can be defined manually (and technically a script could be written to do this automatically)
- disk-space hog
However, although not suited to large-scale ecommerce, or data-critical environments where trends between datasets need to be analysed, or pretty much anything involving binary data, the XBASE format is more than suitable for:
- storage of repetitive text data, especially long strings such as HTML pages, comma-delimited data, and so forth
- small to medium-scale ecommerce application, especially since Miva comes with its own Miva Merchant ecommerce add-on
- small to medium templatising of web pages via data drawn from a database
- quick and dirty password protection via databased information
The author has used Miva in most of these scenarios, including one where Miva was helped out in processing the databased information by Perl, resulting in an efficient text search of a 2 million record database in only a few seconds. Miva has also provided a complete solution to dynamically presenting the information for an entire website in two languages, including the handling of HTTP upload of this information, and the managing of the users who have permissions to do this. This whole site is managed with 3 scripts.
Miva databases can be thought of as being handled in two stages: creation, utilisation and maintenance.
Creating databases is via, unsurprisingly, MVCREATE, and is as follows:
<MVCREATE NAME="db" DATABASE="my.dbf" FIELDS="name CHAR(30), sex BOOL, heightcm NUMBER(3.2), dob DATE, notes MEMO"> Here, all field types have been shown:
- CHAR, which can be up to 250 characters long, an alphanumeric storage field
- BOOL, a logical on/off field
- NUMBER, which takes an argument saying how many places there are before and after the point; here, the heightcm field gives the person's height to two decimal places
- DATE which stores the date in epoch seconds. Note that the field does not store time information, one big failing of the XBASE format
- MEMO, which stores an amount of text limited only by storage and processor capacity and capability
Once the database is created, you can find out what the original information to create it was via the MVREVEALSTRUCTURE command, which creates its own little database with one record per field of the original database. More likely though, you'll want to index the data in some way, and this is achieved as follows.
As any database developer will tell you, the organisation of your data predisposes your indexes to take one form or another, and in most cases careful attention to how you organise your data can reduce or even remove the need for indexes altogether. However, given the proviso that Miva's indexes are limited in length (the expression used to generate them cannot be longer than 100 characters, e.g.), and they are restricted in other ways as well - two fields cannot be indexed together in opposite directions, thus indexing in reverse date order with the records then indexed in ascending ASCII order can't be done - indexes are created by naming an index with the id of the open database and creating a file by an idex expression based on the fields within the database:
<MVMAKEINDEX NAME="db" INDEXFILE="my_name.mvx" EXPR="{name}" FLAGS="DESCENDING">
This is a very simple example. If you had a function 'getsurname()' which then looked up the name in another database and returned the person's surname as well, you could just as easily write:
<MVMAKEINDEX NAME="db" INDEXFILE="my_full_name.mvx" EXPR="{getsurname(name)}" FLAGS="DESCENDING">
Your only downside here would be that the function is called for every record, which might be a problem with processor speed, but technically since the index is a true Miva EXPR command, then anything that can be evaluated can be placed there.
So much for creating the database and the index. Now to use the information. First we open the database, either without indices, leaving everything in physical order, i.e. as the records were added:
<MVOPEN NAME="db" DATABASE="my.dbf">
or, we open the database with an index:
<MVOPEN NAME="db" DATABASE="my.dbf" INDEXES="my_name.mvx">
or, in some cases, we might want to open the database without an index, and then apply one later on:
<MVOPEN NAME="db" DATABASE="my.dbf">
... more code here ...
<MVSETINDEX INDEXES="my_name.mvx">
We can also access the database with as many 'views' as we wish, simply by using a different alias (name) each time:
<MVOPEN NAME="plaindb" DATABASE="my.dbf">
<MVOPEN NAME="indexdb" DATABASE="my.dbf"
INDEXES="my_name.mvx">
Accessing the 'plaindb' alias will not utilise the index, whereas using indexdb will. When accessing the data, Miva provides a few, powerful, commands to access the information:
- MVFIND will search on a keystring across the indexed fields, matching from left-right; thus, searching for 'fred' would find 'fred flintstone'
- MVGO will move around the database physically, i.e. ignoring any index if present.
- MVSKIP acts like MVGO but follows the logical, indexed order, not the physical order the database is created in
- MVFILTER will provide a 'view' of the database based upon a filter expression, e.g.:
<MVFILTER FILTER="{last_name EQ 'morris'}">
All matching records will be kept, and the others 'hidden', although if the data was then read into a new database (copying the structure of the original via the use of MVREVEALSTRUCTURE) it could literally filter the data into a new database, and discard or archive the old. Note that if an index is open on the filtered database, then the filter will leave this ordering intact.
Lastly, after accessing the information, we are also going to want to maintain it.
Miva provides three commands to directly manipulate the information:
- adding data is achieved with MVADD
- editing data is done with MVUPDATE
- removing data is done with MVDELETE (and putting it back uses MVUNDELETE)
and two commands to indirectly alter its status in the database as a whole:
- MVREINDEX will reset the open index files
- MVPACK will remove deleted records permanently and also tidy up MEMO fields (which reside in a separate file outside the main database)
All indexing is operational only on open indexes, and more than one can be opened for this purpose:
<MVOPEN NAME="db" DATABASE="my.dbf" INDEXES="my_name.mvx,my_full_name.mvx">
(note however that when accessing data, only the first index will actually apply, so there is little point in opening more than one when not actually altering the database).
This sums up the quick overview of databases, and with it, most of this introduction to Miva. There remain now just a few pointers on where to find more information, scripting examples, and help.
Annex
Of course, the first place to start is http://www.miva.com itself, where demos of the personal server edition of Miva can be had for free registration and full documentation is available for all flavours of Miva.
Of the several scripting sites, Mr Moon's website at http://mrmoon.com.
There is also a very helpful and friendly mailing list, which can be joined by emailing miva-users@miva.com with:
subscribe listname youremail
as the first line in the body text (leave subject blank)