How to save a region of interest on a QGraphicsScene

[Ballas]

Hi all

I am developing a Image editor using QGraphicsScene. The QGraphicsScene contains two QGraphicsItems. One is a large QGraphicsPixmapItem as a background item and a another one is small QGraphicsRectItem as a foreground object. The foreground object movable. The QGraphicsRectItem acts like a region of interest selector for the image loaded in the background. I want to save only the region of interest as seen under that QGraphicsRectItem when I release the mouse handle.

Currently I am getting the original scene being saved instead of the updated position of the rect item.

Can someone point me in the right direction please.

[SGaist]

Hi and welcome to devnet,

Can you share the code you are currently using ?

[Ballas]

Hi SGaist,

Thanks for the warm welcome and the reply. The code that I am using is attached below

#include <QGraphicsScene>
#include <QGraphicsView>
#include <QWidget>
#include <QGraphicsSceneMouseEvent>
#include <QVBoxLayout>
#include <QGraphicsPixmapItem>

class ImageViewerScene : public QGraphicsScene
{
    Q_OBJECT
public:
    explicit ImageViewerScene(QObject *parent = 0);

protected:
    void mousePressEvent(QGraphicsSceneMouseEvent *mouseEvent) override;
    void mouseMoveEvent(QGraphicsSceneMouseEvent *mouseEvent) override;
    void mouseReleaseEvent(QGraphicsSceneMouseEvent *mouseEvent) override;
};

class ImageViewer : public QWidget
{
    Q_OBJECT
public:
    explicit ImageViewer(QWidget *parent = nullptr);
    void setImage(const QString & fileName);
    void addOverlay(QRect rect);

private:
    QGraphicsView *graphicsView;
    ImageViewerScene *scene;
};

ImageViewer::ImageViewer(QWidget *parent) : QWidget(parent)
{
    scene = new ImageViewerScene(this);
    graphicsView = new QGraphicsView(scene);

    QVBoxLayout *topLayout = new QVBoxLayout;
    topLayout->addWidget(graphicsView);
    setLayout(topLayout);

    graphicsView->setDragMode(QGraphicsView::NoDrag);
}

void ImageViewer::setImage(const QString & fileName)
{
    QImage image(fileName);
    QGraphicsPixmapItem *pixmap = new QGraphicsPixmapItem(QPixmap::fromImage(image));
    graphicsView->setSceneRect(0, 0, image.width(), image.height());

    scene->addItem(pixmap);
}

ImageViewerScene::ImageViewerScene(QObject *parent) : QGraphicsScene(parent)
{
}

void ImageViewerScene::mousePressEvent(QGraphicsSceneMouseEvent *event)
{    
    QGraphicsScene::mousePressEvent(event);
}

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{
    QGraphicsScene::mouseMoveEvent(event);
}

void ImageViewerScene::mouseReleaseEvent(QGraphicsSceneMouseEvent *event)
{
    QGraphicsScene::mouseReleaseEvent(event);
}

void ImageViewer::addOverlay(QRect rect)
{
    QPen redPen(Qt::red);
    redPen.setWidth(2);

    QGraphicsRectItem *rectItem = scene->addRect(QRectF(rect), redPen);
    rectItem->setFlag(QGraphicsItem::ItemIsMovable, true);
}

The two main items on the sene are the image and the overlay. The image is a larger background object in the scene and the overlay is a smaller foreground object.

I wanted to know what is the best approach is access the image section under the overlay rectangle. Ideally I want to get a QImage pertaining to this section of the image and apply some image processing filters. Also the image processing filters will be updated when the mouse events on the scene occur for the selected foreground overlay rectangle.

[SGaist]

Do you want to apply the processing in place ?

In any case, one possibility is to grab the coordinates of the overlay, then use the original image, do you processing and then replace the content of the QGraphicsPixmapItem containing the image.

[Pl45m4]

@Ballas

Do you want to use your image processing on your ROI only? So basically a new cropped image from your large image?

You can create a new QImage based on the ROI like this:
(It will take X, Y, width and height from your rectItem)

QImage roiImage = image.copy(rectItem->rect()); 
// Maybe you have to map the rect to correct coords first

Check out this and this.

[Ballas]

Thanks for the replies.

Yes, I want to perform operation in place and for every mouse move operation.

I will try out the suggestions and post back on how it goes.

[Ballas]

Hi

The mousePress and mouseMove event handlers for the sccene is implementated as below.

void ImageViewerScene::mousePressEvent(QGraphicsSceneMouseEvent *event)
{    
    auto currentItem = itemAt(event->scenePos(), QTransform());
    if(currentItem != nullptr)
    {
        auto itemRect = currentItem->boundingRect();
        qDebug() << "Pressed at :  Mouse " << event->scenePos() << " Rect : "
                 << itemRect;
    }

    update();
    QGraphicsScene::mousePressEvent(event);
}

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{
    auto currentItem = itemAt(event->scenePos(), QTransform());
    if(currentItem != nullptr)
    {
        auto itemRect = currentItem->boundingRect();
        qDebug() << "Moved to :  Mouse " << event->scenePos() << " Rect : "
                 << itemRect;
    }

    update();
    QGraphicsScene::mouseMoveEvent(event);
}

Is it fair to expect that the when the user uses the mouse to update the GraphicsItem position on the scene,
the graphics items has its bounding rect updated. From the logs, I see that the updates position of the graphics items do not point to the correct location.

Pressed at :  Mouse  QPointF(597,401)  Rect :  QRectF(499.5,299.5 201x201)
Moved to :  Mouse  QPointF(596,401)  Rect :  QRectF(499.5,299.5 201x201)
Moved to :  Mouse  QPointF(595,401)  Rect :  QRectF(499.5,299.5 201x201)
Moved to :  Mouse  QPointF(595,401)  Rect :  QRectF(499.5,299.5 201x201)
Moved to :  Mouse  QPointF(594,401)  Rect :  QRectF(499.5,299.5 201x201)
Moved to :  Mouse  QPointF(594,401)  Rect :  QRectF(499.5,299.5 201x201)

[SGaist]

You might want to do these calculation after the base class call rather than before.

[Ballas]

I have tried the following

1. Gather the updated position after base class update

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{
    QGraphicsScene::mouseMoveEvent(event);
    auto currentItem = itemAt(event->scenePos(), QTransform());
    if(currentItem != nullptr)
    {
        auto itemRect = currentItem->boundingRect();
        qDebug() << "Moved to :  Mouse " << event->scenePos() << " Rect : "
                 << itemRect;
    }
}

2. Update the item, call the base class and gather the updated position after base class update

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{  
    auto currentItem = itemAt(event->scenePos(), QTransform());
    currentItem->update();
    QGraphicsScene::mousePressEvent(event);
    if(currentItem != nullptr)
    {
        auto itemRect = currentItem->boundingRect();
        qDebug() << "Moved to :  Mouse " << event->scenePos() << " Rect : "
                 << itemRect;
    }  
}

3. Update the item, gather the updated position and then call the base class handler

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{  
    auto currentItem = itemAt(event->scenePos(), QTransform());
    currentItem->update();  
    if(currentItem != nullptr)
    {
        auto itemRect = currentItem->boundingRect();
        qDebug() << "Moved to :  Mouse " << event->scenePos() << " Rect : "
                 << itemRect;
    }  
  QGraphicsScene::mousePressEvent(event);
}

The position of the graphics item remains intact in all the cases. I guess i am missing from fundamentals about how the graphics item position is handled on the scene.

[Asperamanca]

@Ballas
This may be an interesting read: Graphics View Coordinate system

The boundingRect of an item is in item coordinates. That means, when the item is moved, only the item's position changes, not it's boundingRect.
If you want an item's bounding rect in scene position, you need to map it to scene coordinates.

[Ballas]

Hi

I am using setRect of the graphics item everytime a mouse move is encountered. Although I am able to achieve what I intended, I am not sure if the method I am using is technically correct.

I will also try the suggestion to map to scene coordinates.

void ImageViewerScene::mousePressEvent(QGraphicsSceneMouseEvent *event)
{    
    auto scenePos0 = event->scenePos();
    auto currentItem = itemAt(scenePos0, QTransform());
    auto currentRectItem = dynamic_cast<QGraphicsRectItem*>(currentItem);
    if(currentRectItem != nullptr)
    {
        auto itemRect = currentRectItem->boundingRect();
        delta_x = scenePos0.x() - itemRect.x();
        delta_y = scenePos0.y() - itemRect.y();
    }

    QGraphicsScene::mousePressEvent(event);
}

void ImageViewerScene::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{
    auto scenePos0 = event->scenePos();
    auto currentItem = itemAt(scenePos0, QTransform());
    auto currentRectItem = dynamic_cast<QGraphicsRectItem*>(currentItem);
    if(currentRectItem != nullptr)
    {
        currentRectItem->setRect(scenePos0.x() - delta_x,
                                 scenePos0.y() - delta_y,
                                 200.0,
                                 200.0);
    }

    QGraphicsScene::mouseMoveEvent(event);
}

void ImageViewerScene::mouseReleaseEvent(QGraphicsSceneMouseEvent *event)
{
    auto scenePos0 = event->scenePos();

    auto currentItems = items(scenePos0);

    QGraphicsItem *currentItem;
    foreach (currentItem, currentItems)
    {
        auto currentRectItem = dynamic_cast<QGraphicsRectItem*>(currentItem);
        if(currentRectItem != nullptr)
        {
            currentRectItem->setRect(scenePos0.x() - delta_x,
                                     scenePos0.y() - delta_y,
                                     200.0,
                                     200.0);

            auto currentBoundingRect = currentRectItem->boundingRect();
            qDebug() << "Moved to Rect : " << currentBoundingRect;

            QImage img(200, 200, QImage::Format_ARGB32_Premultiplied);
            QPainter imagePainter(&img);
            render(&imagePainter, QRectF(), currentBoundingRect);
            imagePainter.end();

            cv::Mat cvImage0 = QtOcv::image2Mat(img);
            cv::cvtColor(cvImage0, cvImage0, cv::COLOR_BGRA2GRAY);
            cv::blur(cvImage0, cvImage0, cv::Size(7,7));
            QImage procImage0 = QtOcv::mat2Image(cvImage0);

            if(overlayPixmap != nullptr)
            {
                removeItem(overlayPixmap);
                overlayPixmap = nullptr;
            }

            overlayPixmap = new QGraphicsPixmapItem(QPixmap::fromImage(procImage0));
            overlayPixmap->setPos(currentBoundingRect.x(), currentBoundingRect.y());
            overlayPixmap->setFlag(QGraphicsItem::ItemIsMovable, true);
            addItem(overlayPixmap);
        }
    }

    delta_x = delta_y = -1.0;
    QGraphicsScene::mouseReleaseEvent(event);
}

[Asperamanca]

Hmmm....what you are trying to do sounds like a job for QRubberBand on the QGraphicsView rather than a temporary QGraphicsItem.

[Ballas]

Hi all

Thanks for all the support and suggestions. I have a better solution and I am posting it here for your review. Please let me know if you find any other improvements that can be made.

#include <QGraphicsScene>
#include <QGraphicsView>
#include <QWidget>
#include <QImage>
#include <QGraphicsSceneMouseEvent>
#include <QVBoxLayout>
#include <QGraphicsPixmapItem>
#include <QGraphicsRectItem>

#include <opencv2/opencv.hpp>
#include <opencv2/imgproc.hpp>

class OverlayRectangle : public QGraphicsItem
{
public:
    enum { Type = UserType + 1026 };
    OverlayRectangle(const QRectF &, cv::Mat &, QGraphicsItem *parent = 0);

    QRectF boundingRect() const override;
    void paint(QPainter *painter, const QStyleOptionGraphicsItem *option,
               QWidget *widget) override;

    int type() const override;

protected:
    void mousePressEvent(QGraphicsSceneMouseEvent *event) override;
    void mouseMoveEvent(QGraphicsSceneMouseEvent *event) override;
    void mouseReleaseEvent(QGraphicsSceneMouseEvent *event) override;

private:
    QRectF rectShape;
    QSizeF dragDelta;
    bool resizeShape;

    cv::Mat & overlayImage;
};

class ImageViewer : public QWidget
{
    Q_OBJECT
public:
    explicit ImageViewer(QWidget *parent = nullptr);
    void setImage(const QString & fileName);
    void addOverlay(QRectF rect);

private:
    QGraphicsScene *imageScene;
    QGraphicsView *imageView;
    QGraphicsPixmapItem *imagePixmap;

    cv::Mat cvImage;

};

#include "image_viewer.h"
#include "cvmatandqimage.h"

#include <QDebug>

OverlayRectangle::OverlayRectangle(const QRectF & cvRect,
                                   cv::Mat & overlayImg,
                                   QGraphicsItem *parent) :
    QGraphicsItem(parent),
    rectShape(cvRect),
    overlayImage(overlayImg)
{
    setFlag(QGraphicsItem::ItemIsMovable, true);
    setFlag(QGraphicsItem::ItemSendsGeometryChanges, true);
    resizeShape = false;
}

QRectF OverlayRectangle::boundingRect() const
{
    return rectShape;
}

void OverlayRectangle::paint(QPainter *painter,
                             const QStyleOptionGraphicsItem *option,
                             QWidget *widget)
{
    Q_UNUSED(option);
    Q_UNUSED(widget);

    try
    {
        QRectF rect0 = mapRectToScene(boundingRect());
        cv::Rect2d cvRect0(rect0.x(), rect0.y(), rect0.width(), rect0.height());

        cv::Mat croppedImage = overlayImage(cvRect0);
        cv::cvtColor(croppedImage, croppedImage, cv::COLOR_BGR2GRAY);
        cv::GaussianBlur(croppedImage, croppedImage, cv::Size(51, 51), 51, 51);
        QImage processedImage = QtOcv::mat2Image(croppedImage);
        painter->drawImage(rectShape, processedImage);
    }
    catch (...)
    {
        return;
    }

    QPen blackPen(Qt::blue);
    blackPen.setWidth(2);
    painter->setPen(blackPen);

    painter->drawRect(rectShape);
    QRectF brRect(rectShape.x() + rectShape.width() - 20,
                  rectShape.y() + rectShape.height() - 20, 20, 20);
    painter->drawRect(brRect);
}

int OverlayRectangle::type() const { return Type;}

void OverlayRectangle::mousePressEvent(QGraphicsSceneMouseEvent *event)
{
    auto currentPos0 = event->scenePos();
    QPointF dragRectBR = mapToScene(boundingRect().bottomRight());
    QRectF dragRect(dragRectBR.x() - 20, dragRectBR.y() - 20, 20, 20);
    if(dragRect.contains(currentPos0))
    {
        dragDelta.setWidth(dragRectBR.x() - currentPos0.x());
        dragDelta.setHeight(dragRectBR.y() - currentPos0.y());
        resizeShape = true;
    }

    QGraphicsItem::mousePressEvent(event);
}

void OverlayRectangle::mouseMoveEvent(QGraphicsSceneMouseEvent *event)
{
    auto currentPos0 = event->scenePos();
    auto sceneRectTL0 = mapToScene(boundingRect().topLeft());

    if(resizeShape == true)
    {
        auto newWidth = currentPos0.x() - sceneRectTL0.x() + dragDelta.height();
        auto newHeight = currentPos0.y() - sceneRectTL0.y() + dragDelta.height();

        if( newWidth > 50 && newHeight > 50 )
        {
            prepareGeometryChange();
            rectShape.setWidth(newWidth);
            rectShape.setHeight(newHeight);
            update();
        }
        return;
    }

    QGraphicsItem::mouseMoveEvent(event);
}
void OverlayRectangle::mouseReleaseEvent(QGraphicsSceneMouseEvent *event)
{
    resizeShape = false;
    QGraphicsItem::mouseReleaseEvent(event);
}


ImageViewer::ImageViewer(QWidget *parent) : QWidget(parent)
{
    imageScene = new QGraphicsScene(this);
    imageView = new QGraphicsView(imageScene);
    imagePixmap = nullptr;

    QVBoxLayout *topLayout = new QVBoxLayout;
    topLayout->addWidget(imageView);
    setLayout(topLayout);

    imageView->setDragMode(QGraphicsView::NoDrag);
}

void ImageViewer::setImage(const QString & fileName)
{
    QImage image(fileName);

    if(imagePixmap != nullptr)
    {
        imageScene->removeItem(imagePixmap);
        delete imagePixmap;
    }

    imagePixmap = new QGraphicsPixmapItem(QPixmap::fromImage(image));
    imageView->setSceneRect(0, 0, image.width(), image.height());
    imageScene->addItem(imagePixmap);

    cvImage = cv::imread(fileName.toStdString().c_str());
}

void ImageViewer::addOverlay(QRectF cvRect)
{
    auto overlayRect = new OverlayRectangle(cvRect, cvImage);
    imageScene->addItem(overlayRect);
}