face_partition 0 and 1 dimensional faces needed for visualizing PMesh final mesh?

[jfdev001]

Question regarding the branch https://github.com/fverdugo/GalerkinToolkit.jl/tree/parallel_two_level_mesh, src/geometry.jl:

To properly visualize the final_mesh::PMesh, I think I need to have the the 0-dimensional and 1-dimensional face partition info. Currently the 2-dimensional face partition info is already available (this is simply the cell_partition variable and it stores the number of 2D faces -- i.e., cells -- on each partition). The 0-dimensional and 1-dimensional face partition info, however, do have overlapping faces. See the image below in which green is the coarse mesh with 4 coarse cells and each coarse cell belongs to a single MPI rank. The 2x2 blue boxes are each a fine mesh, and there is one fine mesh per coarse cell. The numbers are ids for the 1-dimensional faces (i.e., edges). The image shows that the 1-dimensional face ids have overlap, and ownership of those face ids is on the rank 2 (i.e., the red circled ids).

So to construct this information, I think I need to do something like the following (of course using map where appropriate):

for each d != num_dims(fine_mesh) # since we assume cell-based partitioning
  for each partition
    for each d-dimensional face on a given partition
    # get ownership of d-dimensional face
    # save this owernship info for this particular partition
    end 
  end
  # construct partitioned indices for this particular dimension-d 
  # sync owernship information across partitions for this particular dimension-d
end

Is this the right idea? Feel free to send me scheduled meeting should you think that would be more productive.

[fverdugo]

You can visualize only the cells, not all faces of all dims, with:

d = gk.num_dims(mesh)
gk.vtk_args(mesh,d)
gk.vtk_physical_faces!(vtk,mesh,d)

Does this solve the problem?

[jfdev001]

The commit 81ba52b appears to solve the problem. Visualizing the final-pmesh-cartesian vtk files in the output directory gives the below image, which I believe matches with what we expect. Cycling through vtk frames with Next Frame in paraview highlight the correct fine meshes in the pmesh. The settings to view the image below require running

julia> include("test/geometry_test.jl")

and dropping the attached text file (should be renamed to .pvsm?) in the output/ directory, opening paraview, and clicking Load State.

final-pmesh-cartesian.txt

[fverdugo]

In fact, you should be able to visualize all faces. The only thing you cannot use is the face_partition of lower dimensional faces when visualizing.

[jfdev001]

This is now possible as of commit 5d4c53e. See attached image showing how for a 4 x 4 coarse mesh distributed over 4 ranks where each rank has a 2 x 2 coarse mesh and each coarse cell in the coarse mesh has a 2 x 2 fine mesh, the ownership of the fine cells is clearly independent on each rank.

[fverdugo]

The trick of a dummy face partition is a good one!

You can create a bit more sophisticated dummy face partition so that you don't need to comment out this loop 5d4c53e#diff-82c56201be7e42ac0e99388b3df8b0c75af84e891207b4e5913112824306aabaR281

You can do eg.

## for dimension d < num_dims(pmesh)
n_own_dfaces = map(m->num_faces(m,d),mesh_partition)
n_dfaces = sum(n_own_dfaces)
dummy_dface_partition = variable_partition(n_own_dfaces,n_dfaces)

This --in fact-- corresponds to the partition if you take the overlapping faces at the interface as distinct objects.

[jfdev001]

Okay, I get it! I will replace my dummy partitions with what you have suggested here. I'll also comment about the fact that this partition is not strictly accurate since it just assumes the 0 and 1 dimensional faces at the interface are non-overlapping, even though this is clearly not the case (see image in original question at beginning).

[jfdev001]

Since one of the objectives is to run extensions/GalerkinToolkitExamples/examples/test/example002_tests.jl with this final mesh, to ensure correct solutions at the interfaces, the ownership data for the 0 and 1-dimensional face partition is still something that is needed, correct? So while that data is not strictly needed to visualize the mesh, for accurate solutions it is.

[jfdev001]

Since one of the objectives is to run extensions/GalerkinToolkitExamples/examples/test/example002_tests.jl with this final mesh, to ensure correct solutions at the interfaces, the ownership data for the 0 and 1-dimensional face partition is still something that is needed, correct? So while that data is not strictly needed to visualize the mesh, for accurate solutions it is.

@fverdugo Regarding the above, during our discussion on 2024-02-20, you mentioned the example only requires node partition information, and not face partition information. However, you did mention that in the general case having the 0D and 1D face partition would be useful. You said that for linear elasticity and poisson equation, the 0 and 1D face partition information is not needed, but perhaps you could briefly write why it might be needed in other cases? This way, we could leave this issue open (and I could rename it appropriately) and add a tag for an enhancement for the future.