General informations

This project is called mmap, after the mmap system call present in UNIX environment

I decided to develop this module to make it possible to have a lot of data available and memorized in a persistent way.
Structured files are very performing if tailored for a specific application but are difficult to manage and an hardly reusable code resource.
A better solution is a virtual memory implementation.

• Up to 4 gigabyte of information can be managed.
• Space can be optimized to real data size.
• All the data can be viewed as a single array and accessed with read/write operations.
• No needing to store after use but just use.
• Can be cached.
• Is highly reusable.

How does it work ?

Every instance is connected to a data file (extension .asd) and to an index file (extension .asi). The former is responsible for containing the data, the latter contains informations relatives to data storing and pages organization.
Access to the data file is achieved using read and write call on the swapping space object. The swap space is responsible for allocating new pages and retrieving old ones from the data file.
Data (both index and real data) are stored in 4 Kbytes pages. The index file is made up of a first page called master index page, where are stored integer referring to positions of other pages inside the index file following the master index, those pages are referred as secondary index pages. Inside a secondary index page integers are stored to locate a data page inside the data file.
The addressing is made by 32 bit words, where the first 20 bits are relative to the page, the latest 12 are relative to the byte inside the page.
The first ten bits of the address are used inside the master index page to locate the position of the secondary index page, the following ten bits are used to locate the data page inside the data file.
By convention a number 0 means that the page is not (yet) present in the addressing space.
Pages are cached by the application, when the user create the instance the number of pages to remain in central memory can be provided.
Cache can be modified dynamically during operations and is digested in a Least Recently Used algorithm.
Inside the swapping space the caches are connected to synching threads that are responsible to keep memory and disk images synched on a timed basis.

BEWARE

Documentation

Since the idea behind mmap is so simple there is not furter documentation that need to be readed beside the introduction.
A browsable API documentation is provided in two form:

• a short version, describing essentials classes
• a long version, describing also all internal classes

Download

You can freely download everything, anyway, you are strongly encouraged to read the disclaimer and license section and to give feedback.

the package	15 KB	everything you need to run mmap
a short documentation	22.4 KB	a short documentation of the essential classes you need to know
a long documentation	72.9 KB	a long documentation of all internal classes, even if you should not care to know
a brief example	5.6 KB	this will show you how to use the package
everithing	135 KB	all the above

Source code

Sorry, I'm not willing to distribute source code until the library is not out of beta stage.
At least the support tools needs to be added to say that enough test was done.
Anyway, if you are so interested in see them drop me a line.

The only scrap of code that can be of interest for programmers is the following:

// file should be truncated to size f

// since jdk 1.2 has RandomAccessFile.setLength() and jdk 1.1.x does not
// here we use a reflection api to runtime check if class has method
// and use it instead of the old "copy and rename" trick
Method m[] = rafData.getClass().getDeclaredMethods();
int i = 0;
while (i < m.length && !(m[i].getName().equals("setLength"))) i += 1;
if (i == m.length) {
    // we are using jdk 1.1.x
    int l, bs = Page.pageSize * 100;
    File fl = new File(rafFileName + ".swp");
    byte buffer[] = new byte[bs];
    FileOutputStream fos = new FileOutputStream(fl);
    rafData.seek(0);
    while (f > 0) {
        f -= (l = rafData.read(buffer, 0, bs < f ? bs : f));
        fos.write(buffer, 0, l);
    }
    fos.close();
    rafData.close();
    fl.renameTo(new File(rafFileName + ".asd"));
    rafData = new RandomAccessFile(rafFileName + ".asd", "rw");
}
else {
    // we are using jdk 1.2
    // setLength can't be called directly or the code will not compile
    // when using jdk 1.1.x
    try {
        Object o[] = new Object[1];
        o[0] = new Integer(f);
        m[i].invoke((Object) rafData, o);
    }
    catch (IllegalAccessException iae) {
        System.err.println("Internal problem!: " +
                           "Illegal access exception");
        System.err.println("Try using jdk 1.1.x");
        iae.printStackTrace();
    }
    catch (InvocationTargetException ite) {
        System.err.println("Internal problem!: " +
                           "Invocation target exception");
        System.err.println("Try using jdk 1.1.x");
        ite.printStackTrace();
    }
}   

This code can be compiled either with jdk 1.1.x and 1.2.x, the behaviour at runtime will be different depending if the setLength facility is implemented for RandomAccessFile or not.

License and disclaimer

I know that it's boring but someone has to say it

This software if freeware, but as many of you know there is freeware and freeware.

Since I like the BSD license I decided to put this software under that license and not under the most common GPL. In case you don't know what the difference is click here.

The main idea is: give credits and use this software as you wish!

Feedback

Like every programmer I like feedback, flames will be redirected to /dev/null but I will enjoy any other comment.

Every feedback will make me aware if there is interest in this library, pushing me to improve performances and add features.

Tell me about part of the documentation that in your opinion should be expanded and give ideas for new features.

Write me if you are using this package as support for your projects, I will make you aware when new releases come out.

Beside this I'm not english mothertongue, so don't be shy to write me about misspelled words, typos and wrong expressions.