【Live2D Explained】 Extra Big News! There has been a revolution in hand-tracking production.#1 (Patreon)
Content
*This article is a translation of an article I published on Pixiv Fanbox on April 29, 2023.
(I split the article into three parts because this time the article was too long and I couldn't upload the article to Patreon properly.)
★Preface.
As is usual, this article is also very long.
If I wanted to explain only the process, it would be shorter, but I also want to explain the principles of why this is done and why it is done, so there are a lot of words.
If there are parts that you already understand what is written here, please skip over them as appropriate.
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
Good afternoon.
I have some extra big news for you all today.
I have been slowly writing articles explaining Live2D for hand-tracking for the past few months, and as those who have read them will know,
Basically, my hand-tracking method is a method of force and brute force, a type of making that requires a lot of man-hours and sacrifices of time in order to improve the quality.
A Fanbox supporter who read our recent hand tracking article contacted us with this. "I've discovered a more simplified method of making arms for hand tracking.
I received this information.
Result,
【Before】
【After】
Very good simplification has been achieved!
To be honest, the previous method took too much time and effort, but I think this method can now be easily adopted by everyone! I mean, I think it's a method that can be used for more than just hand-tracking.
(*However, in a sense this is a bug technique for Live2D, so although I have confirmed that it works with VTS, I do not guarantee that it will work with other applications or games. I think it is better not to use it in commercial projects where the specifications have been decided to some extent... especially in game-related projects. I have also confirmed that it works the same on Cubism 5alpha version as well as Cubism4 for now, but I don't know what will happen in the future).
Discussed with Temikeru-san (https://twitter.com/temikerurero) who provided this information,
He kindly agreed to explain this technique on my Fanbox first.
I can't thank you enough for sharing such an amazing technique with us ><.
I've heard from Temikeru-san on Twitter that he will be distributing sample cmo3 files of the arms, so I'll put the link to that at the end of this article.
If you didn't get the full story from this article, please follow Temikeru's Twitter and download the data!
Now, let's get to the explanation!
ーーーーーーーーーーーーーーーーーーーーーーーーーーーーーー
Firstly, on how much could be simplified.
Here are the previous and previous articles.
As reported in the article above, basically I controlled the angle by hitting a large number of dots on the rotating deformer, just like in the video below.
Particularly at the shoulder (and elbow) X and Y values around 0, it is necessary to dot so finely that no space between the dots is visible, and if moved slowly, the rougher points will be more noticeable.
However, after the modification, it has been freed from this large number of dots, and also from any roughness of movement of the shoulder XY around value 0.
It is a tremendous improvement, don't you think?
(The Gif is very small, but don't worry, it will be explained later in a larger image.)
How on earth do they control such a magical angle adjustment?
↑If you've noticed it in the video up top, you're very perceptive,
This movement is a combination of 'blendshape' and 'move a rotation deformer with warp deformer'.
I know it doesn't make a lot of sense to you to say 'move a rotation deformer with a warp deformer', but I will explain it to you now so you don't have to worry.
(Review) What is a blendshape?
I've given a course on YouTube about what blendshapes are, but I haven't talked about them much in Patreon, so I'll briefly explain them again.
Blendshapes are special parameters that can be used to combine movements.
*About Blendshape (Youtube).
For example, let's say you have added hair swaying movements to two regular parameters like this.
As shown in the video above, the two movements are not linked when just adding the shaking.
In order to combine these two movements, one must create the movements oneself using methods such as "Automatic Generation of Four Corner Forms".
But with blendshapes, this "work of combining the two movements" is not necessary.
(Using blendshapes, the two parameters are automatically linked together.)
In this way, if you create movements, they are automatically multiplied together.
It has made it really easy to create a swinging movement.
There is one more innovative feature of blendshapes.
It is "Limit Settings for Blend Shape Weights”
For example, "I made disgusted eyes with a blendshape, but the closed eyes look wrong...".
If this is a normal parameter, you can directly adjust the shape of the closed eye, but blendshapes "automatically" multiply the movements, so you cannot change the shape of only a part of the blendshape.
However, with 'Limit Settings for Blend Shape Weights' it is possible to 'adjust the effect of the blendshape's deformation only in certain areas'.
''Limit Settings for Blend Shape Weights' takes the form of managing graphs like this.
The graph weights range from 0% to 100%, with 100% being "as-is with blendshape deformation";
the closer you get to 0%, the less deformation is applied, and 0% is "completely without blendshape deformation".
So, the way to deal with this case of "the disgusted eyes has gone wrong when the eyes closed" is to "set the weight value of the disgusted eyes to 0% only when the eyes is closed"
With the ← eye, select the part that was deformed during the disgusted eye and open the blendshape's weight settings.
Create a graph with Right Eye Open/Close,
When the right eye open/close is 0 (closed eye), weight 0%.
When the right eye is between 1 and 1.2 (open eye), the weight is 100%.
*I changed the values of the eye open/close parameter from the default state, so that the eyes open at 1 and open further at a value of 1.2.
1 (・ω・) 1.2( ºωº )
If you are making the blink as usual at 0-1, you can ignore the value at 1.2.
If the → eye is set up in the same way, the movement will no longer be wrong when the eye is closed!
This is a fairly rough explanation, but this is the basic use of blendshapes.
Incidentally, in the current version (Cubism4), only warp deformers and art mesh movements can be connected to blendshapes with deformations.
The movement, drawing order and transparency of rotation deformers will be available from Cubism 5.
This time, however, it is only the warp deformer that is connected to the blendshape.
Next, I will explain another key technique of the method, how to 'move a rotation deformer with a warp deformer'.