## Chapter 7. File I/O

## Overview

The file I/O library in X-Forge is called `XFcFile`. It has been modeled after the standard C library `fopen()`/`fclose()` style file handling. No C++ file stream extensions are included.

In addition to file I/O, the library makes sure all the data is always found in the same directory as where the binary executable lies on all of the platforms, and also transparently wraps compressed file library I/O as well. The library also handles any file system inconsistencies (such as slash vs backslash for directory separation).

For portability, applications should always use slash separator for directories and keep all filenames case sensitive. All of the supported platforms can handle long filenames, but in order to keep possible porting problems minimal, it is recommended that filenames are kept under 32 characters long, and they should only include alphabets (a-z only), numbers and underscores, followed by a point and a three letter extension.

## Standard File Access Calls

### Opening Files

The files are opened, and `XFcFile` object is created, using the XFcFile class static member `open()`. The syntax is as follows:

    XFcFile *f = XFcFile::open(XFCSTR("filename.dat"),XFCSTR("rb"));

Since the two parameters are of CHAR type, the static data strings ("filename.dat" and "rb") are enclosed in `XFCSTR()` macro, which will take care of ASCII-unicode conversion should one be needed.

### Note

Currently all of the platforms supported by X-Forge are internally ASCII. The decision whether we should change all of the platforms to unicode to ease any possible localization effort is still pending. For the time being it is safe to remove the `XFCSTR()` macros, but if the change to unicode is made, all of this code will break. The compiler will alert if this happens, though.

The filename can include directories, but the 'root' directory is still the directory where the executable is. Internally, the core adds the current directory to the filename before passing it to the operating system. Thus, `filename.dat` turns into `/my documents/filename.dat`. Obviously, if you try to open `../filename.dat` it will turn into `/my documents/../filename.dat`, which most probably won't work. If you absolutely need to access files outside the application's directory, you can bypass the filename mangling completely by starting the filename with two slashes (for example, `//c:\config.sys`). This code will not be portable across platforms, however.

The files can exist in either on the disk as separate files (in the same directory as the executable, or its subdirectories), or inside compressed file library (CFL) files. The system first checks files on disk, and then inside CFLs, in order to make development easier. In order for the system to access CFL files they must first be mounted using `XFcCore::openCFL()` call.

If you want to force the system to open from disk or from a CFL, you can use `XFcFile::openDiskFile()` or `XFcFile::openCFLFile()` calls instead of the normal `open()`.

### Warning

Currently, when you open a file that is inside a CFL, the whole file will be uncompressed at that time into memory. This is necessary in order to support random access with the way that CFL is currently implemented. Keeping many files open and/or using large files can eat a lot of runtime memory. If you need to stream a lot of data, it is best not to use CFL for that.

File access modes work as follows:

    "r"  - Open for reading. If file does not exist, opening fails.
    "w"  - Open for writing. If file existed, its contents are destroyed.
    "a"  - Open for appending. Opens file and sets the write head at the end
           of the file. If the file does not exist, it is created.
    "r+" - Opens file for reading and writing. File must exist before the call.
    "w+" - Same as r+, except that if file existed its contents are destroyed.
    "a+" - Same as r+, except that it creates the file if it doesn't exist, and
           write head is positioned at the end of the file.
    "b"  - Consider the data to be binary, and don't do any character conversion.

Files that are inside a CFL file are read-only, so if the file access flag contains w, a or +, only files on disk are accessed.

### Warning

Due to the inconsistency of ASCII-mode file I/O, it is highly recommended that all files are always opened using binary mode.

Files inside CFLs never do any ASCII-mode conversions.

### Closing Files

Files can be closed, and `XFcFile` object freed, by using the `XFcFile` class function `close()`.

    f->close();

Any memory allocated by opening the file is also freed.

### Seeking Inside a File

File seeking is done via the `XFcFile` class function `seek()`.

    f->seek(offset, seek type);

The offset is the amount of bytes to move in either direction.

The following seek types are supported:

    SEEK_CUR - Current position in file
    SEEK_END - End of file
    SEEK_SET - Start of file

In order to find out where the file pointer is at the moment, you can use the `tell()` call.

    INT32 position = f->tell();

### Reading Data

Data can be read either byte by byte using `getChar()` call, or in a chunk using `read()` call. The `getChar()` function represents the `fgetc()` call on stdlib, but due to the fact that `getc()` is implemented as a macro on some platforms we cannot use the same name.

Syntax for `getChar()` is as follows:

    INT32 data = f->getChar();

Please note that `getChar()` does not return `CHAR` type data (which could be 8 or 16 bit depending on whether the platform is unicode), but 8-bit integers. Another return value can be EOF, which is outside the 8-bit range (it is defined by `stdio.h` as (-1)).

Syntax for `read()` is as follows:

    INT32 count = f->read(targetBuffer, blockSize, blockCount);

The returned value is the number of blocks read. Usually programs ignore this information, using file size information instead of the return value to figure out how much data is available. The number of bytes read is blockSize * blockCount.

### Writing Data

Data can be written using the `putChar()` or `write()` functions. The syntax for `putChar()` is as follows:

    INT32 result = f->putChar(value);

The return value will be EOF if there was an error, or the same value as written if not.

The syntax for `write()` is as follows:

    INT32 count = f->write(buffer, blockSize, blockCount);

The returned value is the number of blocks written. If blockCount is not equal to count, the system ran out of disk space.

### Renaming Files

Files can be renamed using the `XFcFile` class static function `rename()`.

    INT result = XFcFile::rename(XFCSTR("old.dat"), XFCSTR("new.dat"));

The result value is 0 if successful, non-zero otherwise.

Trying to rename files that are inside a CFL will fail.

### Warning

Removing or renaming a CFL file that has been mounted may cause undefined behavior.

### Deleting Files

Files can be deleted using the `XFcFile` class static function `remove()`.

    INT result = XFcFile::remove(XFCSTR("unneeded.dat"));

The result value is 0 if successful, non-zero otherwise.

### Warning

While `remove()` can be used to destroy directories in some platforms, this functionality may not work on others.

Trying to delete files that are inside a CFL will fail.

### Warning

Removing or renaming a CFL file that has been mounted may cause undefined behavior.

## File Information

### Checking If File Exists

You can check whether a file exists using the `XFcFile` static function `fileExists()`.

    INT result = XFcFile::fileExists(XFCSTR("myfile.dat"));

The result is 0 if file does not exist, 1 if it can be found inside a CFL, and 2 if the file can be found on disk. If file can be found in both places, the function will return 2, since files are first accessed from disk.

### Finding Files

File finding can be done using the `XFcFileFind` class. The class can be used to enumerate through CFL and disk file directories.

The syntax is as follows:

    XFcFileFind *ff = XFcFileFind::create(filemask, flags);
    CHAR *filename = ff->next();
    while (filename != NULL)
    {
        // do something with filename
        filename = ff->next();
    }
    delete ff;

File mask supports DOS-style file masking, ie. you can use * and ? wildcards. The ? wildcard matches one character while the * wildcard matches any number of characters. "fo?.dat" matches "foo.dat" but not "fooo.dat"; "fo*.dat" matches both "fo.dat" and "fooooooooo.dat". Unlike DOS, you can use several * wildcard characters, so "f*o*o*.dat" matches correctly to "foobar.dat" and ignores "figaro.dat" correctly.

Different flags can be used to alter the `XFcFileFind`'s behavior.

    XFCFF_DISK - find files on disk directory
    XFCFF_CFL - find files inside CFL directory
    XFCFF_REMOVEDUPLICATES - remove duplicates (if the same file exists on both disk and CFL)
    XFCFF_IGNORECASE - perform case-insensitive file name matching

### Warning

Some platforms iterate through disk directories and others don't.

### Getting File Size

You can get the size of a file using the `XFcFile` class static function `getFileSize()`.

    INT32 size = XFcFile::getFileSize(XFCSTR("myfile.dat"));

The size is 0 for files that don't exist. If file of the same name exists both inside CFL and on disk, the size of the disk file is returned.

### Getting Opened File Name

The `XFcFile` object also stores the opened file name. This can be accessed through the `XFcFile` object's `getFilename()` method.

    const CHAR *name = f->getFilename();

Please note that the pointer will become invalid once the file object is closed (and thus destroyed).

## Data Serialization Extensions

The `XFcFile` class includes a bunch of data serialization extensions which are meant to be used in any data file use. Any platform-specific data conversion is handled by these functions as well.

Please note that the `REAL` data type is never read or written directly, as its internal specification may change. Such data should always be stored as `FLOAT32`.

    INT32 writeFLOAT32(FLOAT32 aValue);
    INT32 writeFLOAT64(FLOAT64 aValue);
    INT32 writeINT32(INT32 aValue);
    INT32 writeINT16(INT16 aValue);
    INT32 writeINT8(INT8 aValue);
    INT32 writeUINT32(UINT32 aValue);
    INT32 writeUINT16(UINT16 aValue);
    INT32 writeUINT8(UINT8 aValue);

Same can be said about reading functions. The `REAL` type always handles conversion itself, so you can use `readFLOAT32()` and `writeFLOAT32()` directly with them.

    FLOAT32 readFLOAT32();
    FLOAT64 readFLOAT64();
    INT32 readINT32();
    INT16 readINT16();
    INT8 readINT8();
    UINT32 readUINT32();
    UINT16 readUINT16();
    UINT8 readUINT8();

The read functions also come in read-to-reference flavor which can be used to mirror the write functions directly. The returned value is the bytes read.

    INT32 readFLOAT32(FLOAT32 &aValue);
    INT32 readFLOAT64(FLOAT64 &aValue);
    INT32 readINT32(INT32 &aValue);
    INT32 readINT16(INT16 &aValue);
    INT32 readINT8(INT8 &aValue);
    INT32 readUINT32(UINT32 &aValue);
    INT32 readUINT16(UINT16 &aValue);
    INT32 readUINT8(UINT8 &aValue);

For string reading and writing, the following functions handle reading and writing of zero-terminated strings. The zero is also written so that the string can be read as well. `CHAR`-strings are always stored as `CHAR16` and the read/write functions handle the conversion to whatever format is used internally. To store the string as 8-bit characters, use `CHAR8` functions instead.

    void writeCHARString(CHAR *aValue);
    void writeCHAR8String(CHAR8 *aValue);
    void writeCHAR16String(CHAR16 *aValue);
    CHAR * readCHARString();
    CHAR8 * readCHAR8String();
    CHAR16 * readCHAR16String();

## Compressed File Library (CFL) Files

### Understanding CFL

Compressed file libraries are meant as frozen, compressed storage for game resources. Once created, they are frozen and cannot be updated.

CFLs do not support streaming, and thus the only way to get data from a CFL is to read a whole file at once. `XFcFile` wraps around this functionality by storing the file inside a RAM buffer and taking care about file pointers in this ram buffer.

CFL can be extended by writing new filters. By default, the only filters supported are 'No compression' and 'ZLib compression'. In addition to compression, CFL supports preprocess and encryption filters. Preprocess filters can be used to transform data into more compressable form, and encryption filters can be used to make data files more secure.

Please note that no amount of encryption will make the data files un-rippable if your program uses the files. You can, however, make the data files uneditable by using some public key encryption algorithm. If you plan to do so, make sure you open the files from CFL and not from disk.

CFL applies the filters also to the directory data, reducing size and optionally enhancing security.

You can also store data files inside a CFL file in uncompressed form, and request `XFcFile` pointer to it, or you can ask for the offset inside the .CFL file and do the seeking yourself.

The core includes, internally, one instance of `XFcCFL` object, which is linked to the `XFcFile` system. You can open CFL files to this object by using the `XFcCore::openCFL()` calls. If you want to, you can create new instances of `XFcCFL` objects, but accessing these files is not as handy as via the `XFcFile` system.

### Note

Some plans on changing the way CFLs work exist, but their implementation is still open.

### Getting a File at Once

You can load a file at once using the XFcFile class static function `getFile()`.

    INT8 *data = XFcFile::getFile(XFCSTR("myfile.dat"));

This will work on both files inside a CFL and also for files on disk. This function is typically used with XFcFile class function `getFileSize()`.

If you use your own CFL object, you can get the file using the CFL object's `getFile()` function:

    INT8 *data = myCFLObject->getFile(XFCSTR("myfile.dat"));

Using the CFL object you can also load the file into a pre-allocated buffer.

    INT32 writtenBytes = getFile(XFCSTR("myfile.dat"), buffer, maxSize);

### Accessing Data Directly Inside a CFL File

If you wish to include some data with your application that is infeasible to compress with CFL (due to the memory buffer problem or otherwise), but still want to keep distribution file count to a minimum, you can store the file inside CFL in uncompressed form and then access it directly by opening the .CFL file and seeking to the file.

`XFcCFL` object supports this with `getFileOfs()` function.

    INT32 offset = myCFLObject->getFileOfs(XFCSTR("myfile.dat"));
    XFcFile *f = XFcFile::open(XFCSTR("mycfl.cfl"), XFCSTR("rb"));
    f->seek(offset, SEEK_SET)
    ..

The above clip can be simplified by using the `getFilePtr()` call:

    XFcFile *f = myCFLObject->getFilePtr(XFCSTR("myfile.dat"));

If you get a file pointer using `getFilePtr()` you must close it using the `dropFilePtr()` call:

    myCFLObject->dropFilePtr(f);

Closing the file pointer directly may result in unpredictable behavior.

### Creating CFL Files at Runtime

You can also create CFL files at runtime. This can be useful when making saved games. If your game objects require several kilobytes of data, it's best to have them compressed.

CFLs are created using the `XFcCFLMaker` class. The creation process follows the following pattern:

    XFcCFLMaker *maker;
    maker = CFLMaker::create(XFCSTR("mysavefile.cfl"));
    maker->store(XFCSTR("thisdata"), thisdata, thisdatasize, compressFlags);
    maker->store(XFCSTR("thatdata"), thatdata, thatdatasize, compressFlags);
    maker->store(XFCSTR("somedata"), somedata, somedatasize, compressFlags);
    maker->finish(compressFlags);

The create call creates the file and initializes the header; store calls compress and store the data and build the directory in memory. Finally, the finish call serializes the directory, compresses it, stores it, updates header, closes the file and destroys the `XFcCFLMaker` object.

|  | X-Forge Documentation Confidential Copyright © 2003 Fathammer