wiki:XmtIO

XMT Text File I/O

The compute nodes on an XMT are not directly connected to disk storage. Access to data on disk is made possible via the Lustre parallel file system using the snapshot library. To load files into an application running on the XMT there are a few steps that need to be done.

Lustre Mount Point

The Lustre file system is often mounted at a directory named /scratch or something similar. It is convenient to create a subdirectory using your username to store your files. Note: You should always backup data on any scratch disk as old data may be erased by sysadmins when the disk gets full.

File System Worker Daemon

In a second terminal, connect to an XMT service node and run the fsworker application. It will output a hex string that should be put into the SWORKER_EP environment variable in the shell from which you intend to run your application.

In second terminal, execute:

$ fsworker
8c00100000

The fsworker application won't exit, just leave it running in the terminal. The number that is printed is the important number which needs to be saved into the SWORKER_EP environment variable.

In first terminal (i.e., the one in which the XMT application will be run)

bash:

export SWORKER_EP = '8c00100000'

cshell:

setenv SWORKER_EP '8c00100000'

Example

This section describes a simple application which loads a text file that contains two numbers per line which are delimited by spaces into an application running on an XMT.

Example Data File

For this example we assume that the Lustre file system is mounted at /scratch.

Put an example file in your directory under the /scratch file system.

/scratch/userid/test.txt:

0 0
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10

Example C++ Program

This is an example C++ program that loads a text file from the file system. The basic idea is to load the text file into a character array, which can be processed using istringstream in a convenient line by line manner. This may be a convenient processing scheme since text files are often very easy to work with.

test_snap_textfile.cpp

#include <cstdlib>
#include <cstring>
#include <string>
#include <sstream>

#include <luc/luc_exported.h>
#include <snapshot/client.h>

using namespace std;

bool load_textfile(char* filename)
{
  if (snap_init() != SNAP_ERR_OK)
  {
    fprintf(stderr, "Error: snap_init() failed.\n");
    return false;
  }

  char* swEP_string = getenv("SWORKER_EP");
  luc_endpoint_id_t swEP = 0;

  if (swEP_string != NULL)
  {
    swEP = strtoul(swEP_string, NULL, 16);
  }

  int64_t stat_err = 0;
  snap_stat_buf statBuf;

  if (!filename || snap_stat(filename, swEP, &statBuf, &stat_err) != LUC_ERR_OK)
  {
    fprintf(stderr, "Error: snap_stat() failed.\n");
    return false;
  }

  int size = statBuf.st_size / sizeof(char);

  char* buffer = (char*) malloc(sizeof(char) * size);

  if (snap_restore(filename, buffer, statBuf.st_size, NULL) != SNAP_ERR_OK)
  {
    fprintf(stderr, "Error: snap_restore() failed.\n");
    return false;
  }

  string line;
  istringstream iss;
  iss.str(buffer);

  while (!getline(iss, line).eof())
  {
    int u, v;
    istringstream lineStream(line);
    lineStream >> u >> v;

    // Do stuff here, save to array or whatever.
    printf("'%d -> %d'\n", u, v);
  }

  free (buffer);

  return true;
}

int main(int argc, char *argv[])
{
  char sfname[256];

  if (argc != 2)
  {
    fprintf(stderr, "Usage: %s <txtfile>\n", argv[0]);
    exit(1);
  }

  strcpy(sfname, argv[1]);

  if (!load_textfile(sfname))
  {
    fprintf(stderr,"Failed to load text file");
    return 1;
  }
  
  printf("Done.\n");

  return 0;
}

Compile and Execute

Compile and execute the program by issuing the following commands:

c++ test_snap_textfile.cpp -o test_snap_textfile
mtarun -m 2 test_snap_textfile /scratch/userid/test.txt

The output should like this:

'0 -> 0'
'1 -> 1'
'2 -> 2'
'3 -> 3'
'4 -> 4'
'5 -> 5'
'6 -> 6'
'7 -> 7'
'8 -> 8'
'9 -> 9'
'10 -> 10'
Done