Classes | |
| struct | heffte::rank_remap |
| Keeps the local rank and the map from the global rank to the sub-ranks used in the work. More... | |
| struct | heffte::ioboxes< index > |
| Pair of lists of input-output boxes as used by the heffte::fft3d. More... | |
Functions | |
| template<typename index > | |
| box3d< index > | heffte::find_world (std::vector< box3d< index >> const &boxes) |
| Returns the box that encapsulates all other boxes. More... | |
| template<typename index > | |
| bool | heffte::match (std::vector< box3d< index >> const &shape0, std::vector< box3d< index >> const &shape1) |
| Compares two vectors of boxes, returns true if all boxes match. | |
| template<typename index > | |
| bool | heffte::world_complete (std::vector< box3d< index >> const &boxes, box3d< index > const world) |
| Returns true if the geometry of the world is as expected. More... | |
| template<typename index > | |
| std::vector< box3d< index > > | heffte::split_world (box3d< index > const world, std::array< int, 3 > const proc_grid, rank_remap const &remap=rank_remap()) |
| Splits the world box into a set of boxes that will be assigned to a process in the process grid. More... | |
| template<typename index > | |
| bool | heffte::is_pencils (box3d< index > const world, std::vector< box3d< index >> const &shape, int direction) |
| Returns true if the shape forms pencils in the given direction. | |
| template<typename index > | |
| bool | heffte::is_slab (box3d< index > const world, std::vector< box3d< index >> const &shape, int direction1, int direction2) |
| Returns true if the shape forms slabs in the given directions. | |
| template<typename index > | |
| std::vector< box3d< index > > | heffte::reorder (std::vector< box3d< index >> const &shape, std::array< int, 3 > order) |
| Returns the same shape, but sets a different order for each box. | |
| template<typename index > | |
| std::vector< box3d< index > > | heffte::maximize_overlap (std::vector< box3d< index >> const &new_boxes, std::vector< box3d< index >> const &old_boxes, std::array< int, 3 > const order, rank_remap const &remap) |
| Shuffle the new boxes to maximize the overlap with the old boxes. More... | |
| template<typename index > | |
| long long | heffte::count_connections (std::vector< box3d< index >> const &new_boxes, std::vector< box3d< index >> const &old_boxes) |
| Counts the number of point-to-point connections between the old and new box geometries. More... | |
| template<typename index > | |
| std::vector< box3d< index > > | heffte::make_pencils (box3d< index > const world, std::array< int, 2 > const proc_grid, int const dimension, std::vector< box3d< index >> const &source, std::array< int, 3 > const order, rank_remap const &remap=rank_remap()) |
| Breaks the world into a grid of pencils and orders the pencils to the ranks that will minimize communication. More... | |
| template<typename index > | |
| std::vector< box3d< index > > | heffte::make_slabs (box3d< index > const world, int num_slabs, int const dimension1, int const dimension2, std::vector< box3d< index >> const &source, std::array< int, 3 > const order, rank_remap const &remap) |
| Breaks the world into a set of slabs that span the given dimensions. More... | |
| template<typename index > | |
| std::array< int, 3 > | heffte::proc_setup_min_surface (box3d< index > const world, int num_procs) |
| Creates a grid of mpi-ranks that will minimize the area of each of the boxes. More... | |
Detailed Description
HeFFTe operates with indexes that are distributed in boxes across the mpi ranks. Several methods help manipulate such vectors of boxes, note that in each instance the order of the boxes in a single vector should always match.
Function Documentation
◆ find_world()
|
inline |
Returns the box that encapsulates all other boxes.
Searches through the world.in boxes and computes the highest and lowest of all entries.
- Parameters
-
boxes the collection of all input and output boxes.
◆ world_complete()
|
inline |
Returns true if the geometry of the world is as expected.
Runs simple checks to ensure that the inboxes will fill the world.
- Parameters
-
boxes is the collection of all world boxes world the box that incorporates all other boxes
The check is not very rigorous at the moment, a true rigorous test will probably be too expensive unless lots of thought is put into it.
◆ split_world()
|
inline |
Splits the world box into a set of boxes that will be assigned to a process in the process grid.
- Parameters
-
world is a box describing all indexes of consideration, there is no assumption on the lower or upper bound of the world proc_grid describes the number of boxes in each dimension
- Returns
- a list of non-overlapping boxes with union that fills the world where each box contains approximately the same number of indexes
◆ maximize_overlap()
|
inline |
Shuffle the new boxes to maximize the overlap with the old boxes.
A reshape operation from an old to a new configuration will require as much MPI communication as the lack of overlap between the two box sets, hence using a heuristic algorithms, in an attempt to find a reordering of the boxes to increase the overlap with the old and new. Also assigns the given order to the result.
- Parameters
-
new_boxes is a set of new boxes to be reordered old_boxes is the current box configuration order is the new order to be assigned to the result boxes
◆ count_connections()
|
inline |
Counts the number of point-to-point connections between the old and new box geometries.
Given a grid factorization of (a, b) the split into pencils can be done as either (1, a, b) or (1, b, a), the heffte::make_pencils() method computes both factorizations and selects the one that will lead to fewer point-to-point communications. This allows the pencil rotations to be done within the rows/columns of the 2D grid as opposed to communicating between all available ranks.
◆ make_pencils()
|
inline |
Breaks the world into a grid of pencils and orders the pencils to the ranks that will minimize communication.
A pencil is a box with one dimension that matches the entire world, a pencils grid is a two-dimensional grid of boxes that captures a three dimensional world box.
This calls heffte::split_world() and then rearranges the list so that performing a reshape operation from the source to the resulting list will minimize communication.
- Parameters
-
world is a box describing all indexes of consideration, it is assumed that the world is the union of the source boxes proc_grid gives the number of boxes to use for the two-by-two grid dimension is 0, 1, or 2, indicating the direction of orientation of the pencils, e.g., dimension 1 means that pencil.size[1] == world.size[1] for each pencil in the output list source is the current distribution of boxes across MPI ranks, and will be used as a reference when remapping boxes to ranks order is the box index order (fast, mid, slow) that will be assigned to the result.
- Returns
- a sorted list of non-overlapping pencils which union is the world box
◆ make_slabs()
|
inline |
Breaks the world into a set of slabs that span the given dimensions.
The method is near identical to make_pencils, but the slabs span two dimensions.
◆ proc_setup_min_surface()
|
inline |
Creates a grid of mpi-ranks that will minimize the area of each of the boxes.
Given the world box of indexes, generate the dimensions of a 3d grid of mpi-ranks, where the grid is chosen to minimize the total surface area of each of the boxes.
- Parameters
-
world is the box of all indexes starting from 0. num_procs is the total number of mpi-ranks to use for the process grid.
- Returns
- the dimensions of the 3d grid that will minimize the size of each box.
