UE Plugin Download

Below are the download links. For UE source code development, you can compile it yourself. Currently, the plugin is only applicable to UE5 version. The prebuild is currently only available for UE5.3, other versions will be supplemented later.

ue plugin source

ue 5.3 plugin prebuild

UE Instructions

1. Create a new project in UE

   Both blueprint and C++ projects are acceptable. If you need to develop the plugin further, you need to create a C++ project. Import the character (the default pose of the character must be T-Pose, not A-Pose, otherwise the arm performance will be abnormal), then open the project folder, create a new folder Plugins, and then put rebocap_unreal_engine_plugin into Plugins. For example, if you create a testV3 project, the overall directory structure is as follows:

   

2. Reopen UE and it will automatically compile [since the source code is released, it should be compatible with all versions]

   For further development and debugging of the plugin, you can use Rider to directly open [name].uproject for development and easy debugging.

   You can use Rider to compile and check for compilation errors. If you use UE to compile automatically and encounter errors, please check Saved/Logs/[name].Log. The Log output by UE generally has Chinese encoding issues, and you may need to adjust the system encoding to UTF-8 to view it properly.

3. Skeleton Binding Steps

   • Click on the character asset Skeleton Mesh, right-click to create a new animation blueprint, and double-click to edit the animation blueprint. [It is recommended to watch the video for better understanding]
   • Right-click to search for Rebocap, select Rebocap Body Pose and create a node, then connect the small person on the right side of the node to the Result of the output pose.
   • In the lower left corner of the blueprint editing page, click the plus sign to create a new variable. The variable type needs to be searched. Search for Rebocap, select RebocapMapData, class reference, then drag the variable into the node RetargetAsset just created in the blueprint. A variable node will be automatically generated. Then click the compile button in the upper left corner.

   • Click on the newly created variable node, then in the default value on the right, click the plus sign to create a new Map asset. It will automatically jump to a new page. On the new page, the user needs to fill in the skeleton mapping. Note that it is recommended to fill in all 24 nodes. You can fill in according to the skeleton names of Avatar.

     The skeleton names can be viewed by clicking the first light blue skeleton person in the top bar of the blueprint page. You can automatically select the skeleton and view the corresponding points of each skeleton. The 24 nodes in Rebocap are standard human skeleton points. The skeleton names in Rebocap are named after the starting point of the skeleton bones. For example, in VRM, the bone named LeftUpperLeg starts at the hip, so in Rebocap, it is named L_Hip. The starting point of LeftFoot is the ankle, so it is named L_Ankle in Rebocap. In Rebocap, L_Foot corresponds to the toes. In the UE naming system, it is generally called ball.

     L Collar Left shoulder bone

     L Shoulder Left upper arm bone

     L Elbow Left lower arm bone

     L Wrist Left hand bone

     L Hand Left middle finger bone [will not drive]

     If there are many bones, you can choose appropriate positions. For example, if there are 6 spine bones, you can select three of them.

   • Return to the blueprint page (the selected skeleton mapping needs to be saved and compiled to take effect), and select the value of the variable node as the newly created Map asset.

   • Compile again and check for Warning. Generally, there will only be 3 Warnings. If a skeleton map is filled incorrectly, a warning will be reported indicating that a certain skeleton was not found.
   • Close the animation blueprint editing window, click on the top Window->Virtual Production->Live Link, then select Source->Rebocap Source->conn [the port is the port number, if the broadcast port number in Rebocap is changed, it needs to be modified here]. If the Rebocap client is open, connect will be in ok status, otherwise it will be in bad status. Additionally, the user must perform motion calibration before data broadcasting starts.

1. Code Explanation

The main motion control related code is in Source\rebocap\Private\rebocap_pose_node.cpp, while others are peripheral related code such as dll calls and livelink. Among them, the Init_Foot_Vertices_And_SkeletalData function is used to obtain the default skeleton position of the character and the vert points, calculating 6 points for each foot for ground contact. Since it is automatically calculated, it may not be precise enough, and users can find the 6 points on the soles of the feet themselves and pass them in for more accuracy.

PS: Larger feet may cause the character to oscillate up and down. For example, in an extreme case, if a character has 2-meter-long feet and the character itself is only 2 meters tall, when tiptoeing and then landing, if the toes need to stay in contact with the ground, the character will definitely bob up and down.

Video Operation Demonstration

There is no sound here, temporarily used, it will be more like later

Meta Human Modified to Tpose

Below is a video tutorial and the corresponding Tpose conversion file package. Please follow the video for operation. The conversion file is currently only applicable to UE5.3, other versions will be supplemented later.

Download