For a better viewing experience, visit our webpage.
This plugin allows you to quickly retarget motions created on the Meshcapade.me platform onto your own characters in Unreal Engine 5. Bodies created on the Meshcapade platform are created using the SMPL core technology, and are thus referred to as SMPL-bodies.
📝 This version of the plugin has only been tested on Unreal Engine version 5.3. It may or may not work on earlier or later versions.
I. Adding the plugin to your Unreal project
Download the Unreal Plugin directly, or grab from our git repo.
📝 Make sure that your Unreal project is closed before proceeding.
Once you have the plugin downloaded, unzip the plugin and put the Plugins folder into the top level of your Unreal project.
📝 For MacOS users, install Xcode before relaunching. When opening your Unreal project for the first time after adding the plugin, a warning will pop up asking you to rebuild missing modules. Click Yes and continue normally.
📝 For Windows and Ubuntu users, there's nothing extra to do.
II. Getting an animation from Meshcapade.me
Currently, there are two ways to get animations from Meshcapade.me:
-
Motion from video - extract the human motion from a video.
-
Motion from text - find a human motion in our library of thousands of motions.
To get an animation from a video, visit the Meshcapade motion from video page. Follow the prompts until you've created an animated avatar.
To search for a motion from our motion library, visit the Meshcapade editor page. On the top right, there is a search box where you can find animation. Once you've found the animation you want, save the avatar into your vault.
III. Downloading the animation
Go to your avatar vault and open the avatar in editor (which contains the motion you'd like to download). Once the avatar is open in the editor, click the Download button located at the top left corner of the page. Download options will then appear. If you're only interested in downloading the motion–which will make the import process faster, make sure that file format is set to .FBX (.OBJ has no motion). and compatibility mode is set to Unreal - no blend shapes. If you want to use Pose Correctives, then make sure compatibility mode is set to Unreal. Click Download avatar.
IV. Importing the .FBX into Unreal
With the .FBX downloaded, import it into your unreal project (File > Import).
Set the skeleton to SK_MeshcapadeBody. If you don't see it, then click the gear next to the search box and make sure Show Plugin Content is checked. In the content browser, SK_MeshcapadeBody is located here: Content/Plugins/Meshcapade/Meshes/SK_MeshcapadeBody.
📝 Make sure that Import Animation is checked.
📝 If you’re interested in the Pose Correctives, you also need to use the check Import Morph Targets.
📝 If you are importing a single frame .FBX, for example if you created a SMPL-body from an image, also check Use T0 As Ref Pose.
V. Retargeting the animation
One thing you may want to do is retarget the motion from the SMPL-body onto the body of your character. To do so, you will need a retargeter. Retargeters require two IK rigs: one for the source body, the SMPL-body in this case, and one for the target body - your character. The Meshcapade Unreal plugin comes with a sample retargeter for the Unreal mannequin, including an IK rig for the SMPL-body and an IK rig for the Unreal mannequin. If you already have a retargeter from the Unreal mannequin to your character, then this will provide a straightforward way to get motion from a SMPL-body directly onto your character.
If you want to make a retargeter, the first thing you need to do is make an IK Rig for your character. In the top right corner of the Content Browser, click on Settings and make sure Show Plugin Content is checked. Then open Plugins > Meshcapade Content > Rigs. Duplicate the IK_Manniquen rig.
If your character's skeleton follows the UE5 character convention, you only need to change the preview skeletal mesh to your character. If you're using a UE4 character, you may need to change the End Bone of the Spine IK chain to spine_03.
For more information on this subject, see the Unreal documentation on IK Rig Animation Retargeting.
Once you have the IK rig for your character created, you can make a retargeter. Right-click in the Content Browser and type retargeter or go to Animation > Retargeting > IK Retargeter. Double-click the newly created retargeter.
A retargeter contains two IK rigs and the relationship between them. Set the two IK rigs in the details panel.
The poses of the two skeletons need to match as well. To do this, change the display mode to Edit Retarget Pose on the top left. Then rotate the bones of the source and/or the target until the angles match one another as closely as possible.
Finally, the chains of the two IK rigs need to be correlated. There are several ways to do this, but if you copied the Unreal mannequin all you need to do is click Auto-Map Chains > Map All (Exact).
For more information on this subject, see the Unreal documentation on IK Rig Animation Retargeting.
Now that the retargeter is built, it can be used to retarget any number of animations between the SMPL-body and the body of your character. To do so, simply right-click on an animation file (it will have a dark green bar in the middle of it), and select Retarget Animation Assets > Duplicate and Retarget Animation Assets/Blueprints.
Select the retargeter you'd like to use.
This process will create a new animation asset for your character. Here's an example of the retargeted animation next to the original animation.
VI. Editing the animation
If the animation doesn't look right on your character, there are two options:
- The first is to go back to the platform and try to get something more suitable. You can go to the motion from video page and upload a different video to get different results, or you can go to the motion from text section of the editor and try to search for a more suitable animation using a different prompt.
- The second thing you can do is edit the animation after the fact, and there are several ways to do this. An easy way to do this which also gives you lots of control, is to use a control rig that has a forward and backward solver.
The control rig that comes with the third-person template, which we also ship with the plugin, is sufficient for this. In the content browser, go to Plugins > Meshcapade Content > Rigs and duplicate the CR_Mannequin_Body rig.
You will have to make edits to this control rig depending on how different your character's skeleton is from the Unreal conventional skeleton.
In the Preview Scene Settings, change the preview mesh to that of your character.
If you are retargeting a UE4 character, for example, you would need to make a few changes to the skeleton. In the Rig Hierarchy, you can rearrange bones by clicking and dragging them. The spine, neck, and hands are in different places in the UE4 and UE5 conventional skeletons. You would need to make the following changes:
- Move
clavicle_l,clavicle_randneck_01to be underneathspine_03 - Move
headto be underneathneck_01 - On both hands, move
index_01,middle_01,ring_01, andpinky_01to be direct children of their correspondinghandbones.
For more information on this subject, see the Unreal documentation on Control Rig.
Once you have a control rig that works with your character, you can bake the animation onto that control rig. Make a new level sequence by right-clicking and typing level sequence or going to Cinematics > Level Sequence. Drag the animation asset into your level, select it, and then in the sequencer select +Track > Actor To Sequencer > <your animation asset>.
Click the + to the right of Animation and add your animation asset.
Right-click on the top node of your animation in the track panel and choose Bake to Control Rig > <your control rig>. Click Create.
Note: the animation will only bake what's between the red and green markers on the timeline. If you want to trim the animation, you can do it by moving the markers.
Select the new control rig layer on the track, right-click, and choose Add Section > Additive. This will allow you to add an animation on top of the existing animation.
At this point, you can spend as little or as much time as you'd like to make the animation do what you want to do. In the example below, I tweak the height of the global control so that the character doesn't clip the ground as much as he bends forward. You could then fix the hands and fingers or, cartoony motion, etc.
Once you're happy with your animation, the last step is to bake it. Right-click on the top node in the track panel, choose Bake Animation Sequence, and save it. Now your edited animation can be used as normal.
VII. Using Pose correctives
Pose correctives allow for real-time calculation of pose-based deformations to SMPL-bodies. They are a complex set of blend shapes that we apply based on the pose of the skeleton. This achieves much more realistic soft tissue deformation than the traditional skinning method. They are also extremely easy to apply to actor blueprints.
In the example above, the body on the left shows the motion with the pose correctives applied,
and the body on the right shows the pose correctives without the motion.
📝 To be able to use pose correctives, make sure you enable Import Morph Targets in the import step when you import a SMPL-body.
To enable pose correctives on blueprint actor, add a skeletal mesh component that contains a SMPL-body, then add the Pose Correctives actor component to the same blueprint.
The example below has two bodies overlapping to illustrate the result further. The orange body doesn't have pose correctives, the textured body does.
For any questions, please contact us.