Helper wrappers around MPI methods

Collaboration diagram for Helper wrappers around MPI methods:

Functions
template<typename index >
ioboxes< index >	heffte::mpi::gather_boxes (box3d< index > const my_inbox, box3d< index > const my_outbox, MPI_Comm const comm)
	Gather all boxes across all ranks in the comm. More...
int	heffte::mpi::comm_rank (MPI_Comm const comm)
	Returns the rank of this process within the specified comm. More...
bool	heffte::mpi::world_rank (int rank)
	Returns true if this process has the me rank within the MPI_COMM_WORLD (useful for debugging). More...
int	heffte::mpi::world_rank ()
	Returns the rank of this process within the MPI_COMM_WORLD (useful for debugging). More...
template<typename vector_like >
void	heffte::mpi::dump (int me, vector_like const &x, std::string const &message)
	Write the message and the data from the vector-like x, performed only on rank me (if positive), otherwise using all ranks. More...
int	heffte::mpi::comm_size (MPI_Comm const comm)
	Returns the size of the specified communicator. More...
MPI_Comm	heffte::mpi::new_comm_from_group (std::vector< int > const &ranks, MPI_Comm const comm)
	Creates a new sub-communicator from the provided processes in comm. More...
void	heffte::mpi::comm_free (MPI_Comm const comm)
	Calls free on the MPI comm. More...
template<typename scalar >
MPI_Datatype	heffte::mpi::type_from ()
	Returns the MPI equivalent of the scalar C++ type. More...
template<>
MPI_Datatype	heffte::mpi::type_from< int > ()
	Specialization to hand the int type.
template<>
MPI_Datatype	heffte::mpi::type_from< float > ()
	Specialization to hand the float type.
template<>
MPI_Datatype	heffte::mpi::type_from< double > ()
	Specialization to hand the double type.
template<>
MPI_Datatype	heffte::mpi::type_from< std::complex< float > > ()
	Specialization to hand the single-precision complex type.
template<>
MPI_Datatype	heffte::mpi::type_from< std::complex< double > > ()
	Specialization to hand the double-precision complex type.

Detailed Description

HeFFTe is using the C-style of API for the message passing interface (MPI). A set of inline wrappers allow for easier inclusion into C++ methods which helps preserve const-correctness and simplifies variable initialization.

Function Documentation

gather_boxes()

template<typename index >

ioboxes<index> heffte::mpi::gather_boxes ( box3d< index > const  my_inbox, box3d< index > const  my_outbox, MPI_Comm const  comm  )

inline

Gather all boxes across all ranks in the comm.

Constructs an ioboxes struct with input and output boxes collected from all ranks.

Parameters

my_inbox	is the input box on this rank
my_outbox	is the output box on this rank
comm	is the communicator with all ranks

Returns

an ioboxes struct that holds all boxes across all ranks in the comm

Uses MPI_Allgather().

comm_rank()

int heffte::mpi::comm_rank ( MPI_Comm const  comm )

inline

Returns the rank of this process within the specified comm.

Parameters

comm	is an MPI communicator associated with the process.

Returns

the rank of the process within the comm

Uses MPI_Comm_rank().

Example:

int const me = mpi::comm_rank(comm); // good C++ style

as opposed to:

int me; // uninitialized variable, cannot be const
MPI_Comm_rank(comm, &me); // initialization takes a second line, bad C++ style

world_rank() [1/2]

bool heffte::mpi::world_rank ( int  rank )

inline

Returns true if this process has the me rank within the MPI_COMM_WORLD (useful for debugging).

Parameters

rank	within the world communicator

Returns

true if the rank of this processor matches the given rank

Useful for debugging, e.g., cout statements can be easily constrained to a single processor, e.g.,

if (mpi::world_rank(3)){
    cout << ....; // something about the state
}

world_rank() [2/2]

int heffte::mpi::world_rank ( )

inline

Returns the rank of this process within the MPI_COMM_WORLD (useful for debugging).

Useful for debugging.

dump()

template<typename vector_like >

void heffte::mpi::dump	(	int	me,
		vector_like const &	x,
		std::string const &	message
	)

Write the message and the data from the vector-like x, performed only on rank me (if positive), otherwise using all ranks.

Very useful for debugging purposes, when a vector or vector-like object has to be inspected for a single MPI rank.

Template Parameters

vector_like

is an object that can be used for ranged for-loop, e.g., std::vector or std::array

Parameters

me	the rank to write to cout
x	is the data to be written out
message	will be written on the line before x, helps identify what x should contain, e.g., result or reference data

comm_size()

int heffte::mpi::comm_size ( MPI_Comm const  comm )

inline

Returns the size of the specified communicator.

Parameters

comm	is an MPI communicator associated with the process.

Returns

the number of ranks associated with the communicator (i.e., size).

Uses MPI_Comm_size().

new_comm_from_group()

MPI_Comm heffte::mpi::new_comm_from_group ( std::vector< int > const &  ranks, MPI_Comm const  comm  )

inline

Creates a new sub-communicator from the provided processes in comm.

Parameters

ranks	is a list of ranks associated with the comm.
comm	is an active communicator that holds all processes in the ranks.

Returns

a new communicator that uses the selected ranks.

Uses MPI_Comm_group(), MPI_Group_incl(), MPI_Comm_create(), MPI_Group_free().

comm_free()

void heffte::mpi::comm_free ( MPI_Comm const  comm )

inline

Calls free on the MPI comm.

Parameters

comm	is the communicator to be deleted, cannot be used after this call.

Uses MPI_Comm_free().

Note that the method would use const_cast() to pass the MPI_Comm (which is a pointer) to the delete method. This circumvents the C-style of API that doesn't respect the fact that deleting a const-pointer is an acceptable operation.

type_from()

template<typename scalar >

MPI_Datatype heffte::mpi::type_from ( )

inline

Returns the MPI equivalent of the scalar C++ type.

This template cannot be instantiated directly, only the specializations are useful. Direct instantiation indicates an unknown conversion from a C++ to an MPI type.

Template Parameters

scalar

a C++ scalar type, e.g., float, double, std::complex<float>, etc.

Returns

the MPI equivalent, e.g., MPI_FLOAT, MPI_DOUBLE, MPI_C_COMPLEX, etc.