- 1 InterfaceWithMatlabViaBundles
- Package: MatlabInterface
- Category: Interface
- Author(s): Jeroen Stinstra
- Status: Supported in latest version
- Version: 3.0
Interactive Matlab Interface: This module accomplishes a number of steps to integrate MATLAB code into SCIRun, (1) it translates a SCIRun object (field/matrix/nrrd) into a matlabarray and puts the object in the current workspace of matlab, (2) it executes MATLAB code that can be provided within the interface, (3) it translates back the matlabarrays in MATLAB's workspace into SCIRun objects. The difference with the normal Matlab module is that this one uses bundles as input and output to allow for a more flexible interface. Since the use of bundles is more complicated, the Matlab module should be used if fields or matrices need to be adjusted. But for more complicated projects, this module allows for more flexibility.
This module is an extension of the Matlab module. Please review the documentation of this module for a more detailed description of the inner workings of the matlab interface. The main difference with the Matlab module is the way data is imported into MATLAB. This module uses the module concepts for transporting data. In short a bundle is a collection of SCIRun objects bundled into one stream. As the number of elements in a stream is only limited by the amount of memory that can be used, far more variables can be transported into and out of the module. However this comes with the price of a less straight forward representation of data in matlab. In this case all input and output matlab arrays are structures whose field can represent any of the SCIRun objects currently supported by the Matlab package. Note the currently colormaps and path streams are not supported, although they can be fit into a bundled stream, it is not available in a matlab representation. Converters for those classes will be implemented if they are needed.
The bundle to matlab converter has some specific options as well. For matrices and nrrds the default behaviour is to convert them directly into a numeric dense or sparse matrix and all the other properties of the SCIRun object are stripped away, whereas fields are represented by a sub structure, with fields as defined by the MatlabFieldWriter. The options 'numeric array' and 'struct array' control how matrices and nrrds are converted. In case of 'numeric array' all nrrds and matrices will be dense or sparse matrices and the additional fields within the SCIRun object will be stripped away, such as axis names etc. This is the default behavior. In case of 'struct array' all the SCIRun Matrix and Nrrd objects will be translated into sub structures with all the additional SCIRun data put in the fields of this sub matrix. This module will always convert a field into a structure.
The matlab to bundle converter is a little bit more complicated since there are more options for conversion. The first parameter controlling the conversion process is 'prefer matrices' or 'prefer nrrds'. This option controls whether a matrix is converted into a nrrd or a matrix. Of course, some objects like sparse matrices can only be represented by a SCIRun Matrix object and will hence be translated so. The same is true for matrices with a dimension higher than two, these can only become nrrds. For the others there are two representations possible, this option just specifies the preference the user has. Note that matrices and nrrds in the Bundled Stream can be inserted as nrrds and read out matrices. One thing one should keep in mind that in the matlab converter the first dimension of a nrrd is assumed to be the rows space and secondly the column space. Some functions expect the opposite order. Unfortunately the order is not specified in the nrrd object. In case you run into unexpected transposed matrices, use the built in function of the BundleGetMatrix and BundleGetNrrd to do these transpose operations.
A second choice is whether the substructure represent bundles or scirun objects like fields, nrrds, and matrices. When a field in matlab is a dense, sparse or a string the conversion is obvious. However when it is a structure it can be split into a sub bundle with all the fields being separate objects or they can represent more complicate objects such as fields. The default option is 'prefer sciobjects', which will first try to see if the fields correspond to any of the object defined in SCIRun. For example when a subarray has a field called 'node' it will be translated into a field. If no suitable conversion can be found it will be translated into a bundle. In case of 'prefer bundles' all substructures will be bundles again. This means sno fields will be generated and the translation will be a bundle with subbundles that contain only nrrds and matrices.
Overview Matlab/InterfaceWithMatlabViaBundles module mechanism
This module launches MATLAB as a separate process under control of SCIRun. The process can be on a remote machine or a local machine. In case the module is running on a remote machine the communication with matlab will be through sockets. In the latter case the entry fields in the middle panel of the GUI will need to be filled out with the IP address (it will automatically do a DNS lookup), the port number and a password. A password is only necessary for a remote MATLAB engine that is configured to require a password when launching MATLAB. The current implementation has an optional password in the matlabegine.rc, if this field is left empty no password check is done. The current implementation does not make use of secure sockets, this implementation is still under development. In order to run multiple MATLAB processes simultaneously different sessions can be launched. Each session on the same machine will share the global workspace in MATLAB and code is being executed sequentially for all modules making use of that session. Hence variables declared in the global workspace can be shared between these matlab modules.
Once matlab is launched the intro message will appear in the MATLAB OUTPUT window. The module needs to be executed in order to launch matlab, but will then remain active until the module is destroyed. Alternatively MATLAB can be launched from the MatlabEngine Status panel in the right lower corner. It can be disconnected using the 'disconnect' button in the same panel. That the MATLAB process is kept alive between executions is done to smoothen the executing of matlab networks and as well to be able to store variables in matlab's workspace for later usage. However before the module is executed the translation table of SCIRun objects to matlab objects needs to be setup. This is accomplished by connecting the SCIRun object to one of the input ports on the module. Note that there are ports for Matrices, Fields and Nrrds. In the translation menu the Field section deals with the translation of fields, the Matrix section with the translation of Matrices and the Nrrd section with the translation of the nrrds. Each line in this translation table refers to one set of input and output ports. First of all the module will need to know how the object should be called in MATLAB, it is going to be a matlab array and thus needs to have a name. Then depending on the SCIRun object the object can be translated into structured arrays or numeric arrays, whose numeric format can be set as well. See the sections below for more details. Then at the end of the line the name of the matlab array that needs to be translated back into a SCIRun object. Here a name is sufficient as the translation process will do the rest. The name can be the same as the input array, but it might refer to another array as well.
The process of running code in MATLAB is accomplished as follows: SCIRun will translate the SCIRun objects into matlab compatible objects and write them in a file and then instructs matlab to read this file. Since all communication is through the stdin of matlab, using files makes sure that the data does not have to be written out in ASCII readable code. The module is smart enough to recognize that it translated objects before. If this is the case it will not do the translation again and it will use the file already generated. When loading data into matlab the objects that were already there with the same name will be overwritten. Subsequently the module will take the code the user entered in the GUI and wrap it in a 'try/catch' environment and execute it in matlab. All output generated on the stdout will be displayed in the module. It will write a tag of when the code starts executing as well one on when the code finished executing. These are markers for the module to keep track of when matlab finishes executing code. Please make sure that your progam does NOT generate output that resembles these markers as it will confuse the Matlab Interface. Once the end marker is encountered, the module will instruct matlab to save the variables in the workspace so they can be read in by SCIRun.
The third window on the bottom will show the current status of the matlab engine. Note: when session 0 is requested a new session number will be assigned to the matlab process, which has not been used before. The new session number will appear in this status window. This option can be used to give each MatlabInterface Module its own matlab process running in the background.
Before the module can be used, SCIRun needs to know how to run matlab on the local machine. This is accomplished by the configuration file 'matlabengine.rc', which will be created in the SCIRun/services directory in your local HOME directory. This file is copied out of the src tree the first time SCIRun is run. This file configures how matlab should be run. Edit the line 'startmatlab=' to instruct SCIRun how matlab should be started from an 'sh'-compatible shell. If matlab can already be found using the PATH settings in the shell launching SCIRun, this line probably does not need to be altered. All other fields in this file refer to running matlab on a remote machine. In case SCIRun will not be able to launch matlab a message will be displayed asking to check the configuration file.
To open a matlab process on the local machine the 'MATLAB ENIGNE ADDRESS' in the GUI needs to have an empty Address and does not require a port. SCIRun will in this case automatically launch matlab locally. In the latter case no password checks are done. The local manager does support multiple sessions. To clarify the word session: a session is a matlab process. When multiple Matlab modules make use of the same session the workspace in matlab is shared. Hence one module can be used to load a large matrix into matlab, whereas another one can be used to iterate over a process with small matrices, while using the big matrix stored by the first one. This will allow for some efficiency improvements.
In order to run matlab on a different machine then SCIRun, a small server program needs to be run on the remote machine. The latter is called 'scirunremote', this program sets up a socket for listening and launches matlab when a request is made on its socket. This program uses the same 'matlabengine.rc' file as the module (though it will look in the local HOME directory of the remote machine). This configuration file can be used to set a password, restrict login to only certain machines in a certain domain. Currently the communication is over an open socket in the future an openssl implementation will be used for secure connections. In order to launch the scirunremote program, type 'scirunremote -port 5678', or which ever port you want to use. The latter program must be run on the remote system and serves as a daemon for starting all kinds of external programs. When launching this application a list of services will be displayed. For the matlab engine to work properly two services need to be switched on: matlabengine and matlabenginefiletransfer. The currently implementation requires the last one even if there is a shared home directory. In the latter case no files are transfered, and only the names of the directories in which temporary files are stored are exchanged. The latter mechanism will reduce the amount of network traffic and hopefully speed up the function of this module.
This section shortly lists the different options that can be set in the configuration file 'SCIRun/services/matlabengine.rc'
START MATLAB: This line describes how a sh-shell should start matlab. This instruction is executed to launch matlab
DISABLE: This will disable the matlabengine in scirunremote. The service cannot be launched and any request for starting the matlabengine will be denied.
PASSWORD: A simple password, as a first line of defense. Better ways will follow.
RHOSTS: A list of machines that are allowed to log into scirunremote to request the matlabengine service. This list should contain all the machines you want to use for running SCIRun. Any machine not on the list will be denied access. If no machine is listed all machines are welcome to log in.
MATLABTIMEOUT: How long should we wait before giving up. Note that matlab often needs a couple of minutes to startup. The time entered here is in seconds.