# SENSEI data adaptor mesh metadata for multiple chunk rectilinear grid

**URL:** https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411
**Category:** SENSEI
**Created:** [February 26, 2020, 10:05am UTC](https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411 "2020-02-26T10:05:09Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![benjaminjeliot](https://discourse.kitware.com/user_avatar/discourse.kitware.com/benjaminjeliot/32/206_2.png) [@benjaminjeliot](https://discourse.kitware.com/u/benjaminjeliot)
#### Post date: [February 26, 2020, 10:05am UTC](https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411/1 "2020-02-26T10:05:09Z")

</div>

Hi everyone,

I’m currently instrumenting a miniapp with rectilinear grids, multiple processes (MPI) and multiple chunks on each process. For example, I might have 64 MPI processes with 9 chunks per process.

How should I map this to a VTK data structure?

- A vtkMultiBlockDataSet dataset containing vtkRectilinearGrid objects?
- A vtkMultiBlockDataSet, containing vtkMultiPieceDataSet objects, containing vtkRectilinearGrid objects?
- Something else?

How should I set up the metadata in the data adaptor?

- Should NumBlocks be the number of MPI ranks, or the number of MPI ranks \* chunks per process
- Should NumBlocksLocal be 1 or number of chunks per process

Thanks for you help!

Jim 😄

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)
#### Post date: [February 26, 2020, 4:37pm UTC](https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411/2 "2020-02-26T16:37:54Z")

</div>

Hi @benjaminjeliot !

A little more information would help; are the 9 rectlinear grids within a process related to each other? Do they overlap? Are there 9 per rank regardless of problem size (i.e., the simulation uses them for 9 different purposes) or is this the distribution you are using on one specific combination of simulation domain and HPC machine?

If all the chunks across all the ranks form a single, large rectilinear grid, you should be able to model the data as a single grid with different extents on each chunk (no multiblock involved). The oscillator miniapp’s [DataAdaptor](https://github.com/Kitware/sensei/blob/master/miniapps/oscillators/DataAdaptor.cpp#L367) is a good pattern to follow if this is the case.

If not, say because the grids hold different field data or overlap, then it might be better for each grid to be exposed as a different mesh by your data adaptor rather than grouping them into a multiblock.

---

<div class="post-metadata">

### Author: ![benjaminjeliot](https://discourse.kitware.com/user_avatar/discourse.kitware.com/benjaminjeliot/32/206_2.png) [@benjaminjeliot](https://discourse.kitware.com/u/benjaminjeliot)
#### Post date: [March 2, 2020, 3:38pm UTC](https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411/3 "2020-03-02T15:38:05Z")

</div>

Hi @dcthomp,

Thanks for your reply! 😄

All the chunks across all the ranks form one rectilinear grid. The grid is first split into MPI ranks, then split again on each process. The number of (sub)chunks on each process is the same for all processes, but can vary from run to run. Every (sub)chunk has a ghost halo.

So are you saying that:

- metadata MeshType should be VTK\_RECTILINEAR\_GRID?
- metadata NumBlocksLocal is number of chunks on each process?

What should I return from GetMesh? Should it be a vtkMultiBlockDataSet containing n\_chunks\_per\_process vtkRectilinearGrids?

Thanks again,  
Jim

---

<div class="post-metadata">

### Author: ![dcthomp](https://discourse.kitware.com/user_avatar/discourse.kitware.com/dcthomp/32/10_2.png) [@dcthomp](https://discourse.kitware.com/u/dcthomp)
#### Post date: [March 2, 2020, 4:25pm UTC](https://discourse.kitware.com/t/sensei-data-adaptor-mesh-metadata-for-multiple-chunk-rectilinear-grid/411/4 "2020-03-02T16:25:18Z")

</div>

> [@benjaminjeliot](#):
>
> All the chunks across all the ranks form one rectilinear grid. The grid is first split into MPI ranks, then split again on each process. The number of (sub)chunks on each process is the same for all processes, but can vary from run to run. Every (sub)chunk has a ghost halo.
> 
> So are you saying that:
> 
> - metadata MeshType should be VTK\_RECTILINEAR\_GRID?
> - metadata NumBlocksLocal is number of chunks on each process?

Yes. Where you see the oscillator return VTK\_IMAGE\_DATA (meshName == “mesh”) or VTK\_UNSTRUCTURED\_GRID (meshName=“ucdmesh”) you should return VTK\_RECTILINEAR\_GRID instead.

> [@benjaminjeliot](#):
>
> What should I return from GetMesh? Should it be a vtkMultiBlockDataSet containing n\_chunks\_per\_process vtkRectilinearGrids?

Yes.