mmap

General informations How does it work? Documentation Download Source code License and disclaimer Feedback

General informations

This project is called mmap, after the mmap system call present in UNIX environment

I decided to develop this module to make it possible to have a lot of data available and memorized in a persistent way.
Structured files are very performing if tailored for a specific application but are difficult to manage and an hardly reusable code resource.
A better solution is a virtual memory implementation.

This package is freeware, for more informations see the license section.

How does it work ?

Every instance is connected to a data file (extension .asd) and to an index file (extension .asi). The former is responsible for containing the data, the latter contains informations relatives to data storing and pages organization.
Access to the data file is achieved using read and write call on the swapping space object. The swap space is responsible for allocating new pages and retrieving old ones from the data file.
Data (both index and real data) are stored in 4 Kbytes pages. The index file is made up of a first page called master index page, where are stored integer referring to positions of other pages inside the index file following the master index, those pages are referred as secondary index pages. Inside a secondary index page integers are stored to locate a data page inside the data file.
The addressing is made by 32 bit words, where the first 20 bits are relative to the page, the latest 12 are relative to the byte inside the page.
The first ten bits of the address are used inside the master index page to locate the position of the secondary index page, the following ten bits are used to locate the data page inside the data file.
By convention a number 0 means that the page is not (yet) present in the addressing space.
Pages are cached by the application, when the user create the instance the number of pages to remain in central memory can be provided.
Cache can be modified dynamically during operations and is digested in a Least Recently Used algorithm.
Inside the swapping space the caches are connected to synching threads that are responsible to keep memory and disk images synched on a timed basis.

BEWARE

no tools for data integrity are provided
they will in a future release.

Documentation

Since the idea behind mmap is so simple there is not furter documentation that need to be readed beside the introduction.
A browsable API documentation is provided in two form:

Programmers need to read only the short one.

Download

You can freely download everything, anyway, you are strongly encouraged to read the disclaimer and license section and to give feedback.

the package15 KBeverything you need to run mmap
a short documentation22.4 KBa short documentation of the essential classes you need to know
a long documentation72.9 KBa long documentation of all internal classes, even if you should not care to know
a brief example5.6 KBthis will show you how to use the package
everithing135 KBall the above

Source code

Sorry, I'm not willing to distribute source code until the library is not out of beta stage.
At least the support tools needs to be added to say that enough test was done.
Anyway, if you are so interested in see them drop me a line.

The only scrap of code that can be of interest for programmers is the following:



// file should be truncated to size f

// since jdk 1.2 has RandomAccessFile.setLength() and jdk 1.1.x does not
// here we use a reflection api to runtime check if class has method
// and use it instead of the old "copy and rename" trick
Method m[] = rafData.getClass().getDeclaredMethods();
int i = 0;
while (i < m.length && !(m[i].getName().equals("setLength"))) i += 1;
if (i == m.length) {
    // we are using jdk 1.1.x
    int l, bs = Page.pageSize * 100;
    File fl = new File(rafFileName + ".swp");
    byte buffer[] = new byte[bs];
    FileOutputStream fos = new FileOutputStream(fl);
    rafData.seek(0);
    while (f > 0) {
        f -= (l = rafData.read(buffer, 0, bs < f ? bs : f));
        fos.write(buffer, 0, l);
    }
    fos.close();
    rafData.close();
    fl.renameTo(new File(rafFileName + ".asd"));
    rafData = new RandomAccessFile(rafFileName + ".asd", "rw");
}
else {
    // we are using jdk 1.2
    // setLength can't be called directly or the code will not compile
    // when using jdk 1.1.x
    try {
        Object o[] = new Object[1];
        o[0] = new Integer(f);
        m[i].invoke((Object) rafData, o);
    }
    catch (IllegalAccessException iae) {
        System.err.println("Internal problem!: " +
                           "Illegal access exception");
        System.err.println("Try using jdk 1.1.x");
        iae.printStackTrace();
    }
    catch (InvocationTargetException ite) {
        System.err.println("Internal problem!: " +
                           "Invocation target exception");
        System.err.println("Try using jdk 1.1.x");
        ite.printStackTrace();
    }
}   

This code can be compiled either with jdk 1.1.x and 1.2.x, the behaviour at runtime will be different depending if the setLength facility is implemented for RandomAccessFile or not.

License and disclaimer

I know that it's boring but someone has to say it


THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

This software if freeware, but as many of you know there is freeware and freeware.

Since I like the BSD license I decided to put this software under that license and not under the most common GPL. In case you don't know what the difference is click here.

The main idea is: give credits and use this software as you wish!

Feedback

Like every programmer I like feedback, flames will be redirected to /dev/null but I will enjoy any other comment.

Every feedback will make me aware if there is interest in this library, pushing me to improve performances and add features.

Tell me about part of the documentation that in your opinion should be expanded and give ideas for new features.

Write me if you are using this package as support for your projects, I will make you aware when new releases come out.

Beside this I'm not english mothertongue, so don't be shy to write me about misspelled words, typos and wrong expressions.

mail me