Animated GIF : QImage & Libav API

xtingray

Looking for more information about the problem, I was studying the Libav source code. Here are the files related to the encoding/decoding GIF format process:

https://github.com/libav/libav/blob/master/libavformat/gif.c
https://github.com/libav/libav/blob/master/libavcodec/gif.c

If you check the code within the method gif_image_write_image() you will see how Libav makes the translation from a RGB bitmap into the GIF encoding calling routines related to the LZW algorithm.

static int gif_image_write_image(AVCodecContext *avctx,
                                 uint8_t **bytestream, uint8_t *end,
                                 const uint8_t *buf, int linesize)

The sws_scale() function is not making the required GIF encoding, so I guess I will have to try another trick.

SGaist

If I read your code correctly, you selected PIX_FMT_YUV420P as output format for sws_scale. The problem I see with this is that this format is not in the list of formats supported by the gif encoder.

You should likely change that for AV_PIX_FMT_RGB8.

xtingray

I set the format value AV_PIX_FMT_RGB8 instead PIX_FMT_YUV420P and the resulting GIF file is still wrong. Any way, running some tests I got an interesting debugging message:

#3 0x00007f082b8cc073 in gif_clut_index (b=<error reading variable: Cannot access memory at address 0x4039001>, g=<error reading variable: Cannot access memory at address 0x4039000>, r=0 '\000') at libavformat/gif.c:189
#4 gif_image_write_image (x1=0, y1=0, pix_fmt=2, linesize=<optimized out>, buf=0x4038b70 "", height=<optimized out>, width=<optimized out>, pb=0x3d90540) at libavformat/gif.c:227 
#5 gif_write_video (s=<optimized out>, size=<optimized out>, buf=<optimized out>, enc=<optimized out>) at libavformat/gif.c:329
#6 gif_write_packet (s=<optimized out>, pkt=<optimized out>) at libavformat/gif.c:341
#7 0x00007f082b916d65 in write_packet (pkt=0x7ffc79f4b4e0, s=0x3d8ff60) at libavformat/mux.c:334
#8 av_write_frame (s=0x3d8ff60, pkt=0x7ffc79f4b4e0) at libavformat/mux.c:384 <p></p>#9 0x00007f082bbd9bdb in TLibavMovieGenerator::Private::writeVideoFrame (this=0x3d9b700, movieFile=..., image=...) at tlibavmoviegenerator.cpp:385

Initially I was doubting if the Libav API was calling the GIF procedures to encode the image as that format require it. Now that I see a reference to the function gif_image_write_image (), I am pretty sure that it does it, the right encoding process is happening! :D

So, I feel that my problem is more specific now: In some way, I am failing to cast the data from the QImage object into the AVPicture variable or maybe I have to make the cast directly to an AVFrame structure. Not sure at all, I will have to try several ways until got the solution.

I would love to find previous reference about Qt and Libav procedures related specifically to the GIF format , but it seems there is not much art state about it.

SGaist

What format are you using for your QImage ?

xtingray

This is the way I initialize the QImage objects:

QSize size(width, height);
QImage image = QImage(size, QImage::Format_RGB32);

I want to make one point clear: I already can create MOV, AVI and MP4 videos from a QImage array using this code:

int size = avpicture_get_size(PIX_FMT_YUV420P, width, height);
uint8_t *pic_dat = (uint8_t *) av_malloc(size);
RGBtoYUV420P(image.bits(), pic_dat, image.depth()/8, true, width, height);
avpicture_fill((AVPicture *)frame, pic_dat, PIX_FMT_YUV420P, width, height)
ret = avcodec_encode_video2(c, &pkt, frame, &got_output);

My problem is ONLY related to the animated GIF format.

PS: In case you want to take a look to the whole class (without the GIF part) -> https://github.com/xtingray/tupi/blob/master/src/plugins/export/libavplugin/tlibavmoviegenerator.cpp

SGaist

That's because these encoders supports PIX_FMT_YUV420P as input which is not the case for gif.

One thing you could do to simplify your life a bit is to use Format_RGB888 for your QImage so you wouldn't even need the conversion.

xtingray

Every day I'm getting closer to the solution. This is the latest GIF file I could create -> http://maefloresta.com/tmp/test.gif
Using this code:

    QImage img = image.convertToFormat(Format_RGB888);

    AVPacket pkt;
    av_init_packet(&pkt);
    pkt.flags |= AV_PKT_FLAG_KEY;
    pkt.stream_index = video_st->index;
    pkt.data = (uint8_t *) img.bits();
    pkt.size = sizeof(AVFrame);

    av_write_frame(oc, &pkt);

As you can see it, the GIF file is not animated. Just the first frame is displayed, but the format is right. I was sneaking around inside the container using a hex editor and the 15 frames I created are there. I am missing some kind of flag or instruction to activate the animated format.

Any suggestion?

PS: I tried to use the function avcodec_encode_video2() to add the frames into the file, but it was unsuccessful. Creating my own packets (AVPacket) was the best approach.

SGaist

I think you have to set the loop private property on the muxer.

xtingray

Finally, I could create an animated GIF for the first time. Nevertheless, I need to adjust the FPS parameter in some point because the animation looks too slow. I am doing it from the AVCodecContext variable, like this:

    int fps = 24;
    AVCodecContext *c;
    c->time_base.den = fps;
    c->time_base.num = 1;

But anyway, it doesn't matter the value I set for the fps variable, the result is always the same (slow). On the other hand, this is the code I use to process every QImage object:

        int got_packet = 0;
        AVPacket pkt;
        av_init_packet(&pkt);
        pkt.data = NULL; // packet data will be allocated by the encoder
        pkt.size = 0;

        QImage img = image.convertToFormat(Format_RGB888);
        avpicture_fill((AVPicture *)frame, img.bits(), AV_PIX_FMT_RGB24, w, h);

        int ret = avcodec_encode_video2(c, &pkt, frame, &got_packet);
        if (ret < 0) {
            tError() << "Error encoding video frame!";
            return false;
        }

        if (got_packet) {
            pkt.stream_index = video_st->index;
            ret = av_interleaved_write_frame(oc, &pkt);
        } else {
            ret = 0;
        }

Any suggestion about how to fix the FPS issue?

SGaist

IIRC I used something along these lines:

st->time_base.num = 1000;
st->time_base.den = framerate * st->time_base.num;

st being the video AVStream for the output context and code.

The final result of the math should be the same but it gave me the correct output.

xtingray

I was wrong! The problem wasn't the FPS parameter. The problem was the size of the GIF files, they are giants! That's the reason the browser was playing them so slowly. I am talking about tens of MBs for just few seconds.

I need to reduce the frames size,. Maybe I have to retake the sws_scale() approach, the good news is that for the first time I have a working (not so efficient) implementation :P

I guess I will have to look for more documentation about either libswscale or the scale filter. Not so sure yet if the solution is in that direction, but I will give it a try.

PS: The lack of documentation about ffmpeg/libav is intimidating :/

SGaist

That's one good news ! :)

What are the size of your input images ?

xtingray

The size of the input images is actually small: 500 pixels x 500 pixels. The array contains only 20 images and the size of the GIF is around 10 MB (unacceptable!).
Some guys from the FFmpeg list told me that the problem is not related to the dimension, but about either the codec or the format of the images. Additionally, I have to play with some filters to reduce the size of the outcome following some recommendations from this handy link: http://blog.pkh.me/p/21-high-quality-gif-with-ffmpeg.html

It seems the challenge is bigger than me, but I will keep trying to implement a better approach. As animated GIF are hot stuff again, I think there is a great potential to create this kind of files from Qt directly.

SGaist

Thanks for sharing ! Looks pretty interesting !

I agree FFmpeg is not the easiest library to use but it is powerful. I'd recommend also digging in the implementation of the tools and examples they provide. You can get some useful hints in there and check the codecs code, you'll find also interesting stuff.

Otherwise, one thing I did once (but I've lost the code) was to write a "dumper" that gets all possible informations from the various data structure of the codecs like the possible image input formats. It gave me some insight that proved useful to tune the application and avoid e.g. useless conversions.

Animated GIF : QImage & Libav API

Qt Developer

Qt Developer

Qt Developer

Qt Developer

Qt Developer

Qt Developer

Qt Developer