CIBC:Documentation:SCIRun:DevManual:PersistentIO

Persistent IO

This chapter explains how SCIRun reads and writes data to and from streams.

Introduction

To maximize their usefulness, all software systems must be able to store data to disk at the end of execution, and retrieve that data later for additional processing. As the SW system becomes larger and more complex, especially if numerous different groups are contributing to it, there needs to be a consistent, powerful, and straight forward strategy for performing this data storage and retrieval. Core/Persistent encapsulates the SCIRun data storage and retrieval philosophy which attempts to provide such a solution to this problem.

Core/Persistent provides a direct and uniform method for saving to disk the complex data structures used by SCIRun. Through its use, a user can save and later retrieve data that is distributed across a large number of classes and sub-classes, and need not worry about manually handling dynamically allocated (and possibly cross referenced) memory.

Programming with SCIRun Persistent data

Because most of the code added to SCIRun will use a large number of complex SCIRun data structures, it is necessary for the developer to be able to manage storing data in a consistent manner. Core/Persistent provides this ability by specifying a set of routines that are implemented by each of the SCIRun data structures. It also allows the user to implement these routines in his or her code and thus have a complete and consistent method of storing and retrieving data.

Many module implementors will need to include routines in their module for saving to and retrieve data from disk. Software engineers who develop parts of the SCIRun core will also many times need to add routines to provide the IO for these Core codes. By using Core/Persistent, programmers will be able to easily define routines to save the data specifically created by their codes as well as save the data stored in any of the SCIRun data structures that they use.

Specifically, programmers will, for the most part, use the Core/Persistent paradigm in "Datatypes" files. Programmers are also encouraged to use Core/Persistent for their own data structures for two reasons: 1) If their data structure is every used by other people (migrated to the "Datatypes" directory) then it will need to use Persistent. 2) Using Core/Persistent encourages the implementor to consider and provide for the ability to handle multiple "versions" of data files, transparently to the rest of the code.

Persistent Data

Persistent Data is a general term for any data that will at some point need to be written to disk, and then, later, read back in for further processing.

Persistent Object

A Persistent Object is a specific instance of "Persistent Data" and include all SCIRun datatypes that have been subclassed from the Persistent class. (Classes found in Core/Datatypes usually inherit from Class Datatype, which in turn inherits from Persistent.) Persistent Objects have the ability to save and restore themselves from disk. The data is stored on disk by specifying the "type" of object that created the data and version of that object.

PersistentTypeID

Each "Persistent Object" has an unique "type id". This type id allows SCIRun to load the object back from disk by first, creating a "new" object of this type, and then telling the object to load its data. "PersistentTypeID"s consist of a string representing the type (similar to the C++ character string that specifies the type), the parent class of the object, and the "maker" function that will be used to create new objects of this type.

For templated types, the "type_name()" function is implemented. This function turns, for example, the type vector<int>, into the string, "vector<int>".

Maker Functions

Maker functions are used to created a new Persistent Object. They are very straight forward consisting of code that allocates an object and then returns a pointer to the object.

Pio Streams

Pio Streams are used by Persistent Objects to read/write data from/to disk. There are several types of Pio Streams, all of which inherit from the base class Piostream. The subclasses of Piostream are: BinaryPiostream, TextPiostream, and GzipPiostream. These correspond to binary output, text output, and zipped output. (The type of output is usually set by the user at the time the data is to be written out.)

Pio() versus io()

Every persistent object has an a member function (and thus needs to implement) the io() function. This function (an example is given below) implements the saving and loading of the data unique to that object. The Pio() functions are not member functions. They are defined for all data types and simply are used to start the saving of the data (see below.)

Examples

In order to make a 'Datatype' (or any data structure for that matter) persistent, you must follow these steps:

• Inherit from class Persistent (or from class Datatype.)

• Add the following to your class:

void    io(Piostream &stream);
static  PersistentTypeID type_id;
 
static const string type_name(int n = -1);
virtual const string get_type_name(int n = -1) const;

private:
static Persistent *maker();
           

To save a persistent object, you simply create a persistent stream:

Piostream * stream = auto_istream( file_name );
   

Then you call the Pio function for the object, with the stream:

Pio( *stream, object );
   

The Pio() function will use the io() function provided by (or, in the case of built in types, provided for) the type.

All Persistent objects need to define their current version:

// Pio defs.
const int MY_CLASS_VERSION = 1;      
   

The version information can be used to successfully read in old data files. Various "new" object data members can be defaulted to a reasonable value for these old data files.

All Persistent objects must implement the io() function. This is the function that will be called by Pio() to save/load the object. This is an example io() function:

void 
MyClass::io( Piostream & stream )
{
  // All input/ouptut starts with this line: 
  #IF THERE IS MORE THAN ONE VERSION
  int version = stream.begin_class("ColorMap", MY_CLASS_VERSION);
  #ELSE IF YOU ONLY HAVE ONE VERSION CURRENTLY
  stream.begin_class( type_name().c_str(), MY_CLASS_VERSION );
 
  // Save/Load Base Class fields first:
  MyBaseClass::io( stream );
  
  if( version > 1 ) {
    Pio( stream, new_data_field_ );
  }

  // Save/Load individual fields:
  Pio( stream, my_data_ );
  Pio( stream, more_data_ );

  // All input/output ends with this line:
  stream.end_class();
 }

---

Go back to Documentation:SCIRun:DevManual