How to properly interpolate Animated frames over a Bezier Curve shape

Question

I have attempted to change the Animation to have a smooth Ease-in/Ease-out per the Bezier curve. However, the outcome is not satisfactory. Very much appreciate your feedback and ideas how to do this better.

The data provided as a sequence of .png files that show an Animation (extracted from GIF). This is included in data_src directory in github. (see link further below).

User interface has 2 windows:

1. Bezier Curve Ease-In, Ease-Out shape.
2. Left sub-window shows original Animated Frames. Right sub-Window shows modified Animated Frames. There is a slider to control the timeline, Play, Pause and deploy the Bezier Curve to Animated Frames (the right sub-window will change accordingly). See Fig 1

When deploy button is clicked, the sequence of .png files is then to be the shaped according to the Bezier shape - ie Ease-In and then Ease-out. Playing the Animation in real time would show the Ease-in, Ease-out motion.

GitHub link:

https://github.com/mrglobal/bezier_easeinout. The IDE used is QT Creator. Code written in C++

For simplicity sake, I have provided a few hard-coded Ease-In and Ease-Out models in bezier_curve.cpp. The code does not include the user interface with control points to allow the user to dynamically create a bezier curve. See Bezier_Curve::deploy_bezier_curve for hard-coded c0, p0, p1, c1 values.

The main body of code shows how I follow the Bezier Curve shape in an attempt to transmute the original Animated sequence of frames to Ease-in/Ease-Out sequence when played in "real" time.

Please see mainwindow.cpp for more detailed explanation of code arrangement and structure. I have added comments quite extensively - hope they are useful. Note I did not optimize the code as ease of understanding.

The interpretation of the Bezier Curve is shown succintly in diagram below. Basically I super-impose the Animated sequence of N frames (in this case 142 frames). Example just shows 4 points for illustration. At each point, the tangent to the Bezier Curve is compared to next point's tangent. Essentially its the dy/dx of tangents along points on the Bezier Curve. Note the first dy/dx at point 0 is the tangent of the point on the Bezier Curve minus begin angle from a linear line between the 2 end points. See Fig [2] below

[2] https://i.stack.imgur.com/FYiHU.png

The dy/dx value at each point is then converted number of frames by this formula:

   (dy/dx in degrees)/(begin angle in degrees)

The reason for denominator is that the begin angle is where Number of Frames that deviate from the norm (the linear line) is 0. Hence it makes sense (at least to me) to have that as denominator to convert dy/dx to number of frames to deviate from norm.

The converted values are stored as: /* * Degrees Skip Index= (19, -4, -3, 0.... * where each number: * +N : Jump to frame N * e.g. +19 : Jump to Frame 19 * -N : Delay forward sequence by extending same frame contents N times * e.g. -4 extend same frame 4 times * 0 : Maintain sequence * */

The next step is to go through this list and reinterpolate the Animated Frames starting from the first frame. Following the example, this transmutes to the Animated Frame sequence:

+19: Frame 0 has the frame image at Frame Index 19

-4 Frame 1, 2, 3, 4 have the frame image at Frame Index 20

-3 Frame 5, 6, 7 have the frame image at Frame Index 21

0 Frame 8 has the frame image Frame Index 22

The outcome is problematic as it does not appear to follow a smooth transition even on a simple Ease-In Bezier Curve. Also when there is an Ease-in AND Ease-out Bezier Curve, this method of left to right Frame Image shifting runs out of runway.

Here is the minimal code:

Frame Class:

#include <QObject>
#include <QWidget>
#include <QImage>

#define NUMBER_FRAMES 142
#define INTER_FRAME_INTERVAL_MSECS 35

class Frame : public QWidget
{
    Q_OBJECT

public:
    explicit Frame(QWidget *parent = nullptr);
    int index;
    int src_index;
    int delta;
    bool overwritten;
    QString filename;
    QImage *image;

signals:

protected:
    void paintEvent(QPaintEvent *event);
};




class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    MainWindow(QWidget *parent = nullptr);
    ~MainWindow();

    Bezier_Curve *bezier_curve;
    QTimer *timer;
    Frame *active_left_frame;
    Frame *active_right_frame;
    QList<Frame *>frame_list;
    QList<Frame *>frame_new_list;

    void setup_bezier_curve();
    void read_in_frames();

public slots:
    void timer_fired();

private slots:
    void on_horizontalSlider_valueChanged(int value);

    void on_pushButton_2_pressed();

    void on_pushButton_3_pressed();

    void on_pushButton_clicked();

private:
    Ui::MainWindow *ui;
};


/*
 * Read into 2 frame lists - frame_list and frame_new_list from known directory.
 * Arrange the Frames positions from these 2 lists on left and right side of MainWindow.
 * frame_list - contains the original frames read from the known directory. Don't modify this
 * frame_new_list - starts off with identical as frame_lsit (sans filename) and is to be modified
 * per the Bezier Curve shape
 */
void MainWindow::read_in_frames()
{
    QString directory = "data_src/";
    QPoint left_pos, right_pos;
    QPoint center = this->rect().center();

    /*
     * Create Frames in frame_list from known directory containing filenames in format "1.png", "2.png", ..."<NUMBER_FRAMEs-1>.png"
     * Note that the NUMBER_FRAMES is the presribed number of frames and last filename is <NUMBER_FRAMEs-1>.png
     */
    Frame *frame;
    for (int i=0; i< NUMBER_FRAMES; i++){
        frame = new Frame(this);
        frame->image = new QImage();
        frame->index = i;
        frame->src_index = i;
        frame_list.append(frame);
        QString file_str = directory + "3_" + QString::number(i) + "#.png";
        QFile filename(file_str);
        if (filename.exists()){
            frame->filename = file_str;
            frame->image->load(file_str);
            frame->setFixedSize(frame->image->width(), frame->image->height());
        }
   }

   //Create Frames in frame_new_list. The index, image and size are identical to frames_list.
   for (int i=0; i< NUMBER_FRAMES; i++){
        frame = new Frame(this);
        frame->image = new QImage();
        frame->index = i;
        frame->src_index = i;
        frame->image->load(frame_list.at(i)->filename);
        frame->setFixedSize(frame->image->width(), frame->image->height());
        frame_new_list.append(frame);
   }

   /*
    * Setup left Frame and right Frame position in MainWindow display
    */
   left_pos.setX(center.x() - frame->rect().width() - 10);
   left_pos.setY(center.y() - frame->rect().height()/2);
   right_pos.setX(center.x() + 10);
   right_pos.setY(left_pos.y());

   //Update frames positions in the 2 lists accordingly
   for (int i=0; i< NUMBER_FRAMES; i++){
        frame_list.at(i)->move(left_pos);
        frame_new_list.at(i)->move(right_pos);
   }
}

Key logic below:

/*
 * Entry point to modify Animated sequence of png files to follow the Bezier
 * Curve Ease-in/Ease-Out
 */
void Bezier_Curve::deploy_bezier_curve(QList<Frame *>frame_new_list)
{
    //Hard coded: Ease-In Bezier Curve
    QPoint p0= QPoint(37,110);
    QPoint c1= QPoint(127,20);
    QPoint c2= QPoint(1884,37);
    QPoint p1= QPoint(1884,37);

    /*
     * Hard coded: S-shaped Ease-In Ease-Out Bezier Curve
     */
    /*
    QPoint p0= QPoint(37,110);
    QPoint c1= QPoint(117,4);
    QPoint c2= QPoint(1775,162);
    QPoint p1= QPoint(1884,37);
    */

    //Setup the Bezier Curve
    this->bezier_path.clear();
    this->bezier_path.moveTo(p0);
    this->bezier_path.cubicTo(c1,c2, p1);

    //Ensure Bezier Curve Window draws the latest Bezier Curve
    this->repaint();

    /*
     * Calculate the dy/dx in degrees of each Frame instance's tangent vs the next Frame's tangent
     * The output is degrees_skip_index_list.
     *   contents of each item in the list:
     *      "accelerate to frame N" (+N)
     *      "slow down N frames" (-N)
     *      "maintain sequence" (0)
     */
    this->calculate_bezier_degrees(false, frame_new_list);

    //Shape Animation according to this->degrees_skip_index_list
    reinterpolate_frames(frame_new_list);

}

/*
 * The gist is to capture the delta between tangent at each point on the original linear line vs the tangent at the next point
 * Essentially the dy/dx of each frame instance's tangent to next higher frame.
 * The dy/dx is then converted to number of frames to jump forward (if value is +), extend current frame's content to next (-), or continue
 * normal frame increment (if value is 0). The conversion is (dy/dx)/begin_angle
 * begin_angle is the tangent's angle when the bezier curve is a linear straight line.
 *
 * The beginning instance's (index=0) dy/dx is obtained by (tangent angle - begin_angle). The number of frames is obtained by (tangnet angle - begin_angle)/begin_angle.
 * The other subsequent number of frames is computed by (next frame tangent angle - previous frame tangent angle)/begin_angle
 */
void Bezier_Curve::calculate_bezier_degrees(bool initialize, QList<Frame *>frame_list)
{
    this->degrees_list.clear();
    this->skip_extend_index_list.clear();

    /*
     * Setup degree_list which is tangent at each point from 1% to 100% along QPainterPath
     */
    QPainterPath *painter_path;
    painter_path = &this->bezier_path;

    for (qreal percent=0.0; percent <= 1.0; percent=percent+0.01){
        //Get the slope (tangent) at specific percentage
        qreal slope_temp = painter_path->slopeAtPercent(percent);

        int res = std::fpclassify(slope_temp);
        switch (res){
                case FP_INFINITE:
                case FP_NAN:
                    slope_temp = 0.0;
                break;
            default:
                   break;
        }

        //From slope, get the angle (in radians) and convert to degrees
        qreal angle = qAtan(slope_temp);
        qreal degree = angle * 180/3.142;
        qDebug() << "degree=" << degree << " radian=" << angle << " Slope=" << slope_temp << " percent=" << percent;

        //Store in degrees list for further processing
        this->degrees_list.append(degree);

        /*
         * If Initalize, store the degree to begin_angle. Note all points along the path is the same degree
         * since the line between the 2 end points is straight, linear line  when setup initially.
         */
        if (initialize)
            this->begin_angle = abs(degree);

    }

    /*
     * Make all degrees positive.
     * The Degrees are mainly negative because the coordinates system is based on (0,0) on the top Left.
     * We will simplify it by making all degrees positive based on the reference axis for the Animation Graph based on
     * (0,0) on bottom left. This makes the slope positive by such reference. Note that we reserve negative value to a different meaning
     */
    QList<qreal>temp_list;
    for (int i=0; i < this->degrees_list.length(); i++){
        if (this->degrees_list.at(i) > 0){
            qDebug() << "Unexpected positive";
            temp_list.append(this->begin_angle);
        } else
            temp_list.append(abs(this->degrees_list.at(i)));
    }
    this->degrees_list = temp_list;

    //Dont need to go further if initialize
    if (initialize)
        return;

    /*
     * Go through each instance of Frame and calculate the number of Frame instances it should
     * "accelerate" (+) or "slow down" (-) or maintain sequence (0)
     */
    int prev_adjusted_index_topath = 0;
    for (int i=0; i < frame_list.length(); i++){
        /*
         * All lists so far are based on 0% - 100% along the QPainterPath between the two end points.
         * This is not the same scale as the number of Frame instances.
         * We will thus build the skip_delta_list based on the latter scale.
         */
        float scale_ratio = (float) 100/this->degrees_list.length();
        float index_temp = i * scale_ratio;
        int adjusted_index_topath = round(index_temp);

        //Make sure adjusted_index_topath does not exceed this->degrees_list
        if (adjusted_index_topath > (this->degrees_list.length()-1))
            adjusted_index_topath = this->degrees_list.length()-1;

        float delta;
        if (i== 0)
            delta = (this->degrees_list.at(adjusted_index_topath) - this->begin_angle)/this->begin_angle;
        else
            delta = (this->degrees_list.at(adjusted_index_topath) - this->degrees_list.at(prev_adjusted_index_topath))/this->begin_angle;

        int skip_index = round(delta);
        this->skip_extend_index_list.append(skip_index);
        prev_adjusted_index_topath = adjusted_index_topath;

    }
    qDebug() << "Degrees List=" << this->degrees_list;
    qDebug() << "Degrees Skip Index=" << this->skip_extend_index_list;

}

/*
 * Reinterpolate the Frames in frame_list to follow the bezier curve shape
 */
void Bezier_Curve::reinterpolate_frames(QList<Frame *>frame_list)
{
    int delta;
    int dst_index = 0;
    int src_index = 0;
    Frame *prv_frame;

    /*
     * skip_extend_index_list contains the delta by which the frames
     * are to be jumped forward to, extended or just to maintain sequence.
     * E.g.
     * Degrees Skip Index= (19, -4, -3, -2, -2, -1, -1, 0....
     *
     * +N  : Jump to frame N
     *      e.g. +19 : Jump to Frame 19
     * -N  : Delay forward sequence by extending same frame contents N times
     *      e.g. -4 extend same frame 4 times
     *  0  : Maintain sequence
     *
     */
    qDebug() << this->skip_extend_index_list;

    for (int i=0; i < this->skip_extend_index_list.length(); i++){

        if (dst_index >= frame_list.length())
            return;

        delta = this->skip_extend_index_list.at(i);

        if (delta > 0){

            if (dst_index >0){
                prv_frame = frame_list.at(dst_index-1);
                if (prv_frame->overwritten){
                    extend_src_delta_times(dst_index, 1, frame_list);
                    debug_frames(frame_list, dst_index);
                    prv_frame = frame_list.at(dst_index);
                    src_index = prv_frame->src_index + delta;
                    dst_index++;
                }
            } else
                src_index = delta + dst_index;

            copy_src_to_dst_frame(src_index, dst_index, delta, frame_list);
            debug_frames(frame_list, dst_index);
            dst_index++;
        } else if (delta < 0){
            extend_src_delta_times(dst_index, delta, frame_list);
            dst_index = dst_index + abs(delta);
            debug_frames(frame_list, dst_index-1);
        } else {
            extend_src_delta_times(dst_index, 1, frame_list);
            debug_frames(frame_list, dst_index);
            dst_index++;
        }
    }
}

Answer 1

Yup, this is fun problem. The main issue here is that Bézier curves are non-linear. One simple option is to simply restrict the 2 middle control points to be 1/3 along in x, and 2/3 along in x respectively. This causes the curve in the x direction to follow a linear curve, which effectively reduces the problem to a 1D Bézier curve in the y axis (i.e. you can plonk 't' in directly to the bezier equation, and everything will pop out correctly)

However, 1D Bézier curves have a somewhat limited usefulness wrt to animation (there are some simple easing curves that cannot be represented with 1D Bézier curves. The only ones that work properly are those whose middle points are linear in x)

When dealing with 2D bezier animation curves, you need to insert a couple of additional restrictions on your data.

Firstly, the curve should be monotonic in x. This is to ensure the curve never loops back on itself (If that were to happen, you could end up with more than one possible value for a given time, which would be bad!).

To force this particular case, ensure that the x coordinates of the middle two points are always greater than p0.x, and less than p3.x. Also ensure that p3.x is greater than p0.x (We don't want an animation going backwards in time).

Typically you start off knowing the x value, but you need to solve for t, in order to solve for y. Pomax (the guy who commented above) has written up an approach for solving this here:

https://pomax.github.io/bezierinfo/#yforx

The solution as described above is pretty much what you want, so I'd recommend starting there (assuming you have the correct restrictions in place, you should have only 1 root between the value of 0 and 1). I'd argue his solution isn't the best option available in this case, but hey, he's provided a detailed explanation and given you source code ;)