InfoOut_mpiFileBuffer Class Reference

Customized buffer to facilitate MPI I/O usage for a single file for all domains within an MPI communicator. More...

#include <infoout.h>

Inheritance diagram for InfoOut_mpiFileBuffer:

Collaboration diagram for InfoOut_mpiFileBuffer:

Public Member Functions
	InfoOut_mpiFileBuffer ()
	Generic constructor is used when no information is provided during declaration. More...
	InfoOut_mpiFileBuffer (const MString &filename, const MString &projectName, MPI_Comm mpiComm=MPI_COMM_WORLD)
	This constructor yields a new instance that can immediately be used to write messages to an MPI file. More...
	~InfoOut_mpiFileBuffer ()
	Destructor calls close() to close the MPI file (if opened) and deletes the MPI write buffer. More...
void	open (const MString &filename, const MString &projectName, MPI_Comm mpiComm=MPI_COMM_WORLD)
	Initialization of the MPI I/O environment. More...
void	close (MBool forceClose=false)
	Closes the MPI file. More...
MInt	setMinFlushSize (MInt minFlushSize)
	Sets the minimum buffer length that has to be reached before the buffer is flushed. More...
Public Member Functions inherited from InfoOut_buffer
	InfoOut_buffer ()
	Generic constructor is used when no information is provided during declaration. More...
virtual MBool	setRootOnly (MBool rootOnly=true)
	Sets interal state of whether only the root domain (rank 0) should write to file. More...
virtual MInt	setMinFlushSize (MInt minFlushSize)
	Sets the minimum buffer length that has to be reached before the buffer is flushed. More...

Protected Member Functions
virtual MInt	sync ()
	Flushes the buffer if flushing conditions are met. More...
virtual void	flushBuffer ()
	Flushes the buffer by writing the contents to the MPI file. More...
Protected Member Functions inherited from InfoOut_buffer
virtual MString	encodeXml (const std::string &str)
	Parses the string input and returns the string with XML entities escaped. More...
virtual MString	getXmlHeader ()
	Return an XML header that should written at the beginning of each log file. More...
virtual MString	getXmlFooter ()
	Return an XML footer that should written at the end of each log file. More...
virtual void	createPrefixMessage ()
	Creates an XML prefix using the domain id that is prepended to each message. More...
virtual void	createSuffixMessage ()
	Creates an XML suffix that is appended to each message. More...
virtual void	flushBuffer ()=0

Private Member Functions
MInt	setMpiWriteBuffer (MInt newBufferSize)
	Delete the current MPI write buffer and allocate a new one. More...

Private Attributes
MInt	m_maxMessageLength
	Maximum message length (excluding formatting) More...
MInt	m_mpiWriteBufferSize
	Size of the MPI write buffer. More...
MChar *	m_mpiWriteBuffer
	MPI write buffer. More...
MBool	m_isOpen
	Stores whether the MPI file was already opened. More...
MString	m_filename
	Filename on disk. More...
MPI_Comm	m_mpiComm
	MPI communicator group. More...
MPI_File	m_mpiFileHandle
	MPI file handle. More...
MPI_Request	m_mpiRequest
	MPI request object (nonblocking I/O) More...

Static Private Attributes
static const MInt	m_maxStringLength = 8192
	Maximum string length (including formatting, default: 4096) More...

Additional Inherited Members
Protected Attributes inherited from InfoOut_buffer
MBool	m_rootOnly
	Stores whether only the root domain writes a log file. More...
MInt	m_domainId
	Contains the MPI rank (= domain id) of this process. More...
MInt	m_noDomains
	Contains the MPI rank count (= number of domains) More...
MInt	m_minFlushSize
	Minimum length of the internal buffer before flushing. More...
MString	m_prefixMessage
	Stores the prefix that is prepended to each output. More...
MString	m_suffixMessage
	Stores the suffix that is apended to each output. More...
std::ostringstream	m_tmpBuffer
	Temporary buffer to hold string until flushing. More...
MString	m_projectName
	Name of the current Project. More...
std::vector< std::pair< MString, MString > >	m_prefixAttributes
Static Protected Attributes inherited from InfoOut_buffer
static const MInt	m_fileFormatVersion = 1

Detailed Description

Author

Michael Schlottke

Date

June 2012

This class can be used as a regular string buffer, as it inherits from stringbuf. On flushing the buffer, the contents of the buffer are written to a file using MPI I/O. The entire MPI communication is hidden from the user, so that the underlying algorithms can be changed/optimized without affecting current implementations.

Internally, an XML file is created to store all messages with each processor's MPI rank so that each message can be attributed to the origin. The XML format ensures that fast and easy-to-develop post processing tools may be used (e.g. extractdomainlog.py). Additionally, meta information about the total number of domains as well as the creation/closing date of the file is saved.

Definition at line 114 of file infoout.h.

Constructor & Destructor Documentation

InfoOut_mpiFileBuffer() [1/2]

InfoOut_mpiFileBuffer::InfoOut_mpiFileBuffer

(

)

Author

Michael Schlottke

Date

June 2012

Definition at line 504 of file infoout.cpp.

  : m_maxMessageLength(0),
    m_mpiWriteBufferSize(0),
    m_mpiWriteBuffer(0),
    m_isOpen(false),
    m_filename(),
    m_mpiComm(),
    m_mpiFileHandle(),
    m_mpiRequest(MPI_REQUEST_NULL) {
  // Nothing here
}

InfoOut_mpiFileBuffer() [2/2]

InfoOut_mpiFileBuffer::InfoOut_mpiFileBuffer	(	const MString &	filename,
		const MString &	projectName,
		MPI_Comm	mpiComm = MPI_COMM_WORLD
	)

Author

Michael Schlottke

Date

April 2012

Internally, this constructor just passes the parameters to open() (see open() for more details).

Parameters

[in]	filename	Filename that should be used for the MPI file.
[in]	mpiComm	MPI communicator for which the file should be opened.
[in]	m_projectName	Projectname given.

Definition at line 526 of file infoout.cpp.

  : m_maxMessageLength(0),
    m_mpiWriteBufferSize(0),
    m_mpiWriteBuffer(0),
    m_isOpen(false),
    m_filename(),
    m_mpiComm(),
    m_mpiFileHandle(),
    m_mpiRequest(MPI_REQUEST_NULL) {
  open(filename, projectName, mpiComm);
}

~InfoOut_mpiFileBuffer()

InfoOut_mpiFileBuffer::~InfoOut_mpiFileBuffer

(

)

Author

Michael Schlottke

Date

April 2012

The MPI write buffer m_mpiWriteBuffer is deleted as well, since it could be that the buffer is allocated using setMinFlushSize() but never opened with open() (and thus not deleted within close()).

Definition at line 545 of file infoout.cpp.

                                              {
  // Close the file
  close();
 
  // Delete write buffer
  delete[] m_mpiWriteBuffer;
}

Member Function Documentation

close()

void InfoOut_mpiFileBuffer::close

(

MBool

forceClose = false

)

Author

Michael Schlottke

Date

April 2012

Any subsequent write statements to the MPI file stream are discarded after this method is called. NOTE: If the MPI file is not closed using this function before MPI_Finalize() is called, an MPI error occurs!

     After close() is called, the MPI root process first writes an XML footer to the file. Then the file is

closed on all ranks.

Definition at line 629 of file infoout.cpp.

                                                  {
  // Only perform finalization if this instance was initialized before
  if(m_isOpen) {
    // Only perform cleanup if forceClose is not set to true
    if(!forceClose) {
      // Force flushing of the internal buffer
      flushBuffer();
 
      // Call MPI_Wait to make sure everything was written to disk
      MPI_Wait(&m_mpiRequest, MPI_STATUS_IGNORE, AT_);
 
      // Make sure that all processes have finished writing, so that the footer can be written before the file is closed
      MPI_Barrier(m_mpiComm, AT_);
 
      // Write root information to log file (only on MPI root)
      if(m_domainId == 0) {
        // Copy buffer string to MPI buffer
        MInt mpiStringLength = getXmlFooter().copy(m_mpiWriteBuffer, m_mpiWriteBufferSize);
 
        // Write to MPI file
        MPI_File_write_shared(m_mpiFileHandle, m_mpiWriteBuffer, mpiStringLength, MPI_CHAR, MPI_STATUS_IGNORE, AT_);
      }
 
      // Close MPI file
      MPI_File_close(&m_mpiFileHandle, AT_);
    }
 
    // Delete MPI write buffer
    delete[] m_mpiWriteBuffer;
 
    // Set pointer to null to avoid problems when the destructor calls delete again
    m_mpiWriteBuffer = 0;
 
    // Reset state variable
    m_isOpen = false;
  }
}

flushBuffer()

void InfoOut_mpiFileBuffer::flushBuffer ( )

inlineprotectedvirtual

Author

Michael Schlottke

Date

June 2012

This method flushes the buffer (i.e. copy the buffer to the MPI write buffer, issue the MPI commands etc.) if the file is open, and if this processor is supposed to write to the file. It ignores any settings for the minimum flush size, and assumes that all string formatting was already performed.

Implements InfoOut_buffer.

Definition at line 714 of file infoout.cpp.

                                               {
  // Only write to MPI if the MPI file was already opened, and only if we're supposed to on this domain
  if(m_isOpen && !(m_rootOnly && m_domainId != 0)) {
    // Before copying the new string into the write buffer, make sure that the previous MPI write has finished
    MPI_Wait(&m_mpiRequest, MPI_STATUS_IGNORE, AT_);
 
    // Retrieve actual string length while copying the string from the temporary buffer to the write buffer.
    // The copy command is invoked with the current buffer size to ensure that there is no buffer overflow.
    MInt mpiStringLength = m_tmpBuffer.str().copy(m_mpiWriteBuffer, m_mpiWriteBufferSize);
 
    // Reset temporary buffer
    m_tmpBuffer.str("");
 
    // Write mpiStringLength characters to file using nonblocking MPI I/O
    MPI_File_iwrite_shared(m_mpiFileHandle, m_mpiWriteBuffer, mpiStringLength, MPI_CHAR, &m_mpiRequest, AT_);
  }
}

open()

void InfoOut_mpiFileBuffer::open	(	const MString &	filename,
		const MString &	projectName,
		MPI_Comm	mpiComm = MPI_COMM_WORLD
	)

Author

Michael Schlottke

Date

April 2012

After a successful call to this method the MPI file stream is ready to use. Any previous information written to the buffer is lost when open is called. This function creates a new file as specified in #filename (deleting any existing files with the same name) and creates the XML prefixes and suffixes used for each message. The root process also writes the necessary XML header information to the file.

Parameters

[in]	filename	Name of the file to open.
[in]	mpiComm	MPI communicator for which to open the file.
[in]	projectName	Projectname given.

Definition at line 566 of file infoout.cpp.

                                                                                                      {
  // Open file only if it was not yet done
  if(!m_isOpen) {
    // Set MPI communicator group
    m_mpiComm = mpiComm;
 
    // Get domain id and number of domains
    MPI_Comm_rank(m_mpiComm, &m_domainId);
    MPI_Comm_size(m_mpiComm, &m_noDomains);
 
    // Set filename
    m_filename = filename;
 
    // Set projectName
    m_projectName = projectName;
 
    // Delete log file if one exists
    MPI_File_delete((MChar*)m_filename.c_str(), MPI_INFO_NULL);
 
    // Set MPI config values
    MInt amode = MPI_MODE_CREATE | MPI_MODE_WRONLY;
 
    // Open MPI file
    MPI_File_open(m_mpiComm, (MChar*)m_filename.c_str(), amode, MPI_INFO_NULL, &m_mpiFileHandle, AT_);
 
    // Clear internal buffer in order to dismiss any previous input
    str("");
 
    // Create prefix and suffix messages and calculate the maximum length for a message
    createPrefixMessage();
    createSuffixMessage();
    m_maxMessageLength = m_maxStringLength - m_prefixMessage.length() - m_suffixMessage.length();
 
    // Create new write buffer with the following size: length before flushing + max MPI string length
    setMpiWriteBuffer(m_minFlushSize + m_maxStringLength);
 
    // Write root and meta information to log file (only on MPI root, i.e. domain id == 0)
    if(m_domainId == 0) {
      // Copy buffer string to MPI buffer
      MInt mpiStringLength = getXmlHeader().copy(m_mpiWriteBuffer, m_mpiWriteBufferSize);
 
      // Write to MPI file
      MPI_File_write_shared(m_mpiFileHandle, m_mpiWriteBuffer, mpiStringLength, MPI_CHAR, MPI_STATUS_IGNORE, AT_);
    }
 
    // Make sure that initial message is written before file is used
    MPI_Barrier(m_mpiComm, AT_);
 
    // Set state variable
    m_isOpen = true;
  }
}

setMinFlushSize()

MInt InfoOut_mpiFileBuffer::setMinFlushSize ( MInt  minFlushSize )

inlinevirtual

Author

Michael Schlottke

Date

June 2012

This calls InfoOut_buffer::setMinFlushSize and then adds some code to change the size of the MPI write buffer m_mpiWriteBuffer along with the minFlushSize.

\params[in] minFlushSize Minimum buffer length.

Returns

The previous value of the minimum flush size.

Reimplemented from InfoOut_buffer.

Definition at line 742 of file infoout.cpp.

                                                                    {
  // Call base class method for general stuff
  MInt previousValue = InfoOut_buffer::setMinFlushSize(minFlushSize);
 
  // Flush current buffer
  flushBuffer();
 
  // Create new write buffer with the following size: length before flushing + max MPI string length
  setMpiWriteBuffer(m_minFlushSize + m_maxStringLength);
 
  return previousValue;
}

setMpiWriteBuffer()

MInt InfoOut_mpiFileBuffer::setMpiWriteBuffer ( MInt  newBufferSize )

private

Author

Michael Schlottke

Date

June 2012

This method calls MPI_Wait to make sure that all write statements have completed before the buffer is deleted. Then m_mpiWriteBuffer is deleted and afterwards recreated with the new buffer size.

\params[in] newBufferSize The new size of the write buffer

Returns

Previous buffer size

Definition at line 765 of file infoout.cpp.

                                                                {
  // Make sure that the write buffer is empty
  MPI_Wait(&m_mpiRequest, MPI_STATUS_IGNORE, AT_);
 
  // Delete old write buffer
  delete[] m_mpiWriteBuffer;
 
  // Create new write buffer
  m_mpiWriteBuffer = new MChar[newBufferSize];
 
  // Save old buffer size to temporary variable and new buffer size to member variable
  MInt oldBufferSize = m_mpiWriteBufferSize;
  m_mpiWriteBufferSize = newBufferSize;
 
  // Return the previous buffer size
  return oldBufferSize;
}

sync()

MInt InfoOut_mpiFileBuffer::sync ( )

protectedvirtual

Author

Michael Schlottke

Date

June 2012

Sync is called automatically when an "endl" is sent to the stream. At first the buffer content is wrapped in the prefix and suffix messages, then the entire string is written to the MPI file using nonblocking communication by calling flushBuffer(). Finally the internal buffers are reset.

Returns

Zero by default.

Definition at line 677 of file infoout.cpp.

                                 {
  // Only write to MPI if the MPI file was already opened, and only if we're supposed to on this domain
  if(m_isOpen && !(m_rootOnly && m_domainId != 0)) {
    // Escape XML entities in buffer
    str(encodeXml(str()));
 
    // Create formatted string and write to temporary buffer
    if(str().length() > (unsigned)m_maxMessageLength) {
      // If buffer content + prefix/suffix message is too long for m_mpiWriteBuffer, use substr and ensure a newline at
      // the end. In substr the '-1' is for the newline character.
      m_tmpBuffer << m_prefixMessage << str().substr(0, m_maxMessageLength - 1) << "\n" << m_suffixMessage;
    } else {
      // Otherwise just copy the whole string
      m_tmpBuffer << m_prefixMessage << str() << m_suffixMessage;
    }
 
    // Only write to MPI file if current buffer length exceeds the minimum size for flushing
    if(m_tmpBuffer.str().length() >= (unsigned)m_minFlushSize) {
      flushBuffer();
    }
  }
 
  // Reset internal buffer
  str("");
 
  // Default return value for sync()
  return 0;
}

Member Data Documentation

m_filename

MString InfoOut_mpiFileBuffer::m_filename

private

Definition at line 121 of file infoout.h.

m_isOpen

MBool InfoOut_mpiFileBuffer::m_isOpen

private

Definition at line 120 of file infoout.h.

m_maxMessageLength

MInt InfoOut_mpiFileBuffer::m_maxMessageLength

private

Definition at line 117 of file infoout.h.

m_maxStringLength

const MInt InfoOut_mpiFileBuffer::m_maxStringLength = 8192

staticprivate

Definition at line 116 of file infoout.h.

m_mpiComm

MPI_Comm InfoOut_mpiFileBuffer::m_mpiComm

private

Definition at line 122 of file infoout.h.

m_mpiFileHandle

MPI_File InfoOut_mpiFileBuffer::m_mpiFileHandle

private

Definition at line 123 of file infoout.h.

m_mpiRequest

MPI_Request InfoOut_mpiFileBuffer::m_mpiRequest

private

Definition at line 124 of file infoout.h.

m_mpiWriteBuffer

MChar* InfoOut_mpiFileBuffer::m_mpiWriteBuffer

private

Definition at line 119 of file infoout.h.

m_mpiWriteBufferSize

MInt InfoOut_mpiFileBuffer::m_mpiWriteBufferSize

private

Definition at line 118 of file infoout.h.

---

The documentation for this class was generated from the following files:

• /home/gitlab-runner/scratch/builds/oxpnswJ6/1/aia/m-AIA/m-AIA/src/IO/infoout.h
• /home/gitlab-runner/scratch/builds/oxpnswJ6/1/aia/m-AIA/m-AIA/src/IO/infoout.cpp