Timeline and Animation API

Lucrecious

I'm trying to integrate Spine into the Godot game engine.

Unfortunately, the AnimationState API provided by Spine, doesn't really fit the engine model/interface very well.

In Godot, there's an animation player that I want to take advantage of, so I'm trying to integrate the Spine animation with that player somehow using the Timeline and Animation API.

The problem is that this API doesn't work as I expected, and I'm getting weird results.

When the walk animation is first played it looks like this:
https://gfycat.com/lameeminentblackbear

However, when I play the run animation and then the walk animation again, then the animation looks like this:
https://gfycat.com/bewitchedfirsthammerheadshark

The legs are messed up and the mouth stays like the running animation.

I tried both the MixBlend_First and MixBlend_Setup enums but neither of them reset the animation.

I'm not really sure what I'm doing wrong.

The code for updating is a simple call to animation.apply on every frame that the animation is updated.

What am I doing wrong?

Nate

When an animation is applied, it only makes changes to the skeleton for the properties it keys. Applying the run animation makes changes to the skeleton. When you play the walk animation after that, the parts of the skeleton that the walk animation does not key retain the changes from when the run animation was applied.

One way to solve this is to key everything at the start of every animation. This isn't a great solution though.

Another way to solve it is to reset the state of the skeleton, for example back to the setup pose. One way to do that is Skeleton setToSetupPose. This is OK, but can wreck smooth transitions between animations. For example, everything that the walk animation doesn't key will instantly snap from the run animation pose to the setup pose.

Another potential solution is to apply the old animation one last time using MixBlend setup, MixDirection out, and alpha 0 to reset everything it keyed. This is similar to what AnimateState does, though if you don't transition from alpha 1 to 0 over time then you'll have snapping just like Skeleton setToSetupPose.

The way AnimationState (which does a lot and is quite a beast) handles it is that when an animation is replaced by another animation, the old animation "mixes out" to the setup pose. This is done by applying the old animation using MixBlend setup and MixDirection out. With those settings, the alpha parameter for Animation apply controls the mixing between the animation (1) and setup pose (0). AnimationState initially uses 1 for alpha and transitions to 0 over time. Note even if you don't use a mix duration for this mixing out transition, the old animation is still applied one last time with alpha 0 to ensure everything it keyed is set to the setup pose. Only when this mixing out is complete does the old animation stop getting applied. While the old animation is mixed out, the new animation is applied on top. This means if the new animation doesn't key all the same properties (like in your spineboy run -> walk scenario), the properties keyed only in the old animation will transition to the setup pose.

Lucrecious

Ah. Thanks again Nate - that's super helpful! I was trying to work backwards from looking at AnimationState, and I couldn't figure out the process for how the animations switch out (kind of hard to parse that out with everything going on). It's a neat piece of code - clever design 🙂

One last question while we are on this Timeline topic. I'm not sure if you can answer this question but I'll give it a shot! In the AnimationState, I was looking at the applyRotateTimeline function, and I'm not sure what it's trying to do to mix the rotations in there. If I wanted to implement my own transition logic, would I need logic like the one in there? The other apply* timelines look pretty normal but what does the applyRotateTimeline do?

I think it would be beneficial if Spine made a few more "tutorials" for the runtime. The currently available ones out there are great and actually encouraged me to write my own module - you guys made it very easy to follow and also implement this stuff, only took me couple days to learn this all. However, something that I thought lacked was mini tutorials with example usage of lower level API. I think it would be cool if, for example, there was a tutorial for a simplified AnimationState, like how there's a tutorial for a simplified Rendering technique. I think it would help a lot with fine tuning integration into engines. I liked your explanation here for example! I'm mainly a developer so learning this animation stuff is pretty new to me.

Thanks

Nate

Cool, glad it was helpful! It's good to hear that things are mostly easy to follow. I don't blame you for finding AnimationState difficult to follow. It's like Perl, even we forget the entire damn thing minutes after looking at it and have to relearn it every time. It has many edge cases and is difficult to unit test.

To explain AnimationState applyRotateTimeline, first some background. The most common type of animation mixing in most software mixes the animations additively using weights which sum to 100%. For example, animation A is applied at 23%, B at 37%, and C at 40%. This is straightforward and works well in many cases, but doesn't allow for everything AnimationState does (mainly layering with multiple "tracks", IIRC). You can do this kind of mixing by using MixBlend add with the Timeline API (it is also possible with AnimationState), but you'll have to manage the weights (alpha parameter for Timeline apply) yourself.

The other MixBlend modes do a kind of mixing that is between the setup and current pose and the timeline's pose. AFAIK Spine is the only software to do this. It allows for some really neat features in AnimationState, but requires that we are able to interpolate between any two poses and that comes with the some new problems. This is easy for most values, such as positions, but if you have 2 rotation values there are two possibilities for what is for example 50% between them, depending on whether you rotate clockwise or widdershins.

One solution is to just choose the direction that results in the smaller rotation. Eg, if you have 0 and 90 and need 50%, you'd use clockwise (45) and not counterclockwise (135). This works fine, except that we are not mixing two static rotations. Our rotations come from animations and represent the rotation of arms/legs/etc that are changing over time. It can happen that for part of the mix duration the shortest rotation between the rotation in the two animations is clockwise, but then changes to counterclockwise. When this happens the interpolated rotation flips nearly 180 degrees, which is almost never what you want. That can look like this:

Loading Image

To solve this, what we do is when we start mixing, we choose the rotation direction based on the shortest rotation. We remember that direction and use it for the rest of the mix. There's also some other logic there to handle when the bones cross, but that's the gist of it. During development we made a nutty little tool to visualize and test some extreme use cases:

Loading Image

If you are doing mixing with weights (MixBlend add), you probably don't have to worry about this. If you are doing mixing between poses, most people use AnimationState and also don't have to worry about this.

I agree it would be nice to have more low level examples. FWIW, one of the few examples we have of the Timeline API is here:
spine-runtimes/MixTest.java at 3.8
Using AnimationState makes this sort of thing so much easier, most people do that. I don't know how Godot works, but I would guess it uses animation weights, so maybe you can go that route.

Lucrecious

Thanks again for the explanation. Really helpful!

I've implemented a similar system to the AnimationState system. In my system for simplicity as a I built it up, I immediately apply the old animation with 0 alpha with setup and mix out right before applying my next animation. Like how you mention here:

Another potential solution is to apply the old animation one last time using MixBlend setup, MixDirection out, and alpha 0 to reset everything it keyed. This is similar to what AnimateState does, though if you don't transition from alpha 1 to 0 over time then you'll have snapping just like Skeleton setToSetupPose.

However, while the hoverboards or different facial expressions switch to the next animation, the rotations (and possibly translations) do not.

Here's literally the c++ code I'm using to apply:

// SpineData is just a container for SkeletonData and AnimationData
// SpineData::apply just calls the apply on the passed in animation with the parameters
void State::apply(const SpineData& spine_data, spine::Skeleton& skeleton) {

// This is only being tested with 1 track so _track.size() == 1.
for (int i = 0; i < _tracks.size(); i++) {
    
    // The tracks that need to be mjxed out
    RotatingTrackVector tracks = _mixing_out[i];
    for (int j = 0; j < tracks.size(); j++) {
        const Track& track = tracks.get(j);
        if (!track.animation) continue;

        // Mix out the track
        spine_data.apply(
            track.animation,
            skeleton,
            track.progress,
            true,
            0, // Immediately mix it out
            spine::MixBlend::MixBlend_Setup,
            spine::MixDirection::MixDirection_Out);
    }

    // Mix in my next track
    const Track& track = _tracks[i];
    if (track.animation) {
        spine_data.apply(
            track.animation,
            skeleton,
            track.progress,
            false,
            1, // Immediately mix it in
            spine::MixBlend::MixBlend_Setup,
            spine::MixDirection::MixDirection_In);
    }

    // Just some clean up
    _mixing_out[i].clean();
}
}

I've tried this code with multiple permutations, like with different blend modes and different orders of application, but nothing seems to work. The animation switches to the next animation but keeps the rotations and (possibly) the translations from the old ones on the non-keyed bones.

Am I doing this right?

Nate

It looks OK, the mixing_out animation should set everything it keys to the setup pose. First I would triple check the animations being applied are the ones you expect. Eg, maybe you mix out the wrong animation, or maybe you continue to apply the mixed out animation after you reset what it keyed to the setup pose. Otherwise I would create an animation with just 1 timeline and step through applying it to see what values are being set. Eg, you should see it take the code path to reset rotation to the setup pose.

You'd likely have an easier time writing an API on top of AnimationState, which may already do most of what you need. You could hide that the API is using AnimationState under the covers and augment it as needed.

Lucrecious

You're right - and I have seriously considered trying to write a wrapper around AnimationState.

The problem is that AnimationState is state-ful and difficult to control when, for example, you want to set an animation at a specific frame every frame.

The way the AnimationPlayer works in Godot is that it interpolates between properties (keys), similar to spine. So I'd like to simply be able to enter in a time for the "top layer" animation instead of doing a .setAnimation() and let it be controlled by the AnimationState.

So the issue is that I want to mix in/out but also control the current animation by setting the time directly (like the Timeline API). To do this, I thought it'd be neat to set the time of the current animation directly, and then use an "update" method similar to AnimationState for mixing in and out.

If I could write a wrapper around AnimationState to do that I would but I can't really think of way to do it without editing AnimationState code or just writing a simpler custom version of AnimationState.

Edit: I thought about it more and looked through the AnimationState API, and I think I got a way to do what I need.

Thanks Nate!