Keep track of number of rectangles drawn over an image using Mouse Events

[duddal_]

Thanks @mrjj

So there is a variable called self.canvas.shapes which is a list.
Every time a rectangle is drawn over the image the list gets updated. It contains the "Shape" object for each rectangle.
For example: [<libs.shape.Shape object at 0x000002638F8FE448>, <libs.shape.Shape object at 0x000002638C89A848>]

I would like to modify the code such that it outputs the following:
[[<libs.shape.Shape object at 0x000002638F8FE448>], [<libs.shape.Shape object at 0x000002638F8FE448>, <libs.shape.Shape object at 0x000002638C89A848>]]

A list of lists so that I can compare the length of the previous and the current.

So in simple terms it should be: test = [[A], [A,B], [A,B,C]]
So that I can now compare len(test[-1]) and len(test[-2])

I have tried looking into canvas.py script but unable to find a solution.

[mrjj]

@duddal_
Hi
So you want a list of lists?

Wont that upset the rest of the drawing code ?

• So that I can now compare len(test[-1]) and len(test[-2])

That will return the size of the lists ?

[duddal_]

Hello @mrjj ,

Yes, it will return the size of the lists.
I am not sure whether it will upset the drawing code, to be honest.

I have a button that changes the brightness of the image. After a rectangle is drawn on the image I push that button. And after every new rectangle is drawn I first have to save the image file (label file). So, for example, The user has drawn a bounding box, and then once the button is pushed a window appears which allows the user to save the image file. And without drawing any more rectangles the button should just change the brightness instead of asking to save the image file again. Therefore I thought if I have a list of lists of the rectangles drawn one after the other, I will be able to compare the size and then it is just the matter of comparing the previous and current size to determine whether a new rectangle has been drawn or not.

I can't quite figure out if there is another way of doing this or not.

[mrjj]

@duddal_

Hi
If you keep your own list, i guess nothing bad will happen :)

But if the goal is only to keep track of rectangles (shapes)
im not sure i understand why you want to keep a list to track it.

Say we have a new variable.
ShapeCountBefore=0;

Then when your SaveButton is pressed
we compare
ShapeCountBefore and Shapes.Size()
if they are the same, no new rect was added.

if ( ShapeCountBefore LESS than Shapes.Size() )
then a new rect was added. and then we
update ShapeCountBefore=Shapes.Size()

if ( ShapeCountBefore BIGGER than Shapes.Size() ) it means
rect was removed

Maybe i dont 100% understand what you want as you say drawn for the rect and not added.

[duddal_]

@mrjj
I am sorry for the late reply!!
So Yes I tried that and after making a few changes it worked. And now I am trying to figure out whether it is possible to save the bounding boxes (which the user draws) in the memory.

So basically as we discussed above, in order to change the brightness of the image, the image needs to be saved, as many times as the new bounding box is created.
I want to eliminate this shortcoming. For example, I want to be able to press the push button and the brightness needs to be changed even though the new bounding boxes are added without saving the image. So If a user draws new bounding boxes then by pressing the Pushbutton, the brightness needs to change, and then when all is done, the user can save the image once.

For this, I thought saving the newly drawn bounding boxes In the memory and then using them would do the trick, but not successful. So I took a look at the canvas.py script to figure out how they are moving the bounding boxes without saving the image(label file), but cannot quite understand that.

What do you think?

Hope you are healthy.
Cheers.

[mrjj]

Hi
Np. welcome back.
Just to be sure. when you say "bounding box", do you mean the Shapes ? ( the shape class)
Also how is the brightness adjusted now ?
Is that by done by the Shape objects ?

Also what do you mean by "how they are moving the bounding boxes without saving the image(label file)"
Where do they move or do you mean like hide ?

[duddal_]

@mrjj
Hello!

1. Yes, "bounding box" means the shape (rectangle).

2. Brightness is adjusted using a push-button. When an image is loaded in the tool and then the user presses the "brightness" push-button, brightness is changed by a certain percentage. And herein lies the issue: When the user draws a new shape over the image then the user needs to save ((Ctrl + S) the corresponding label file (XML format) first. If not saved, then the original image file without shapes or previous shapes is loaded (So to simplify: if any new shape is drawn, it needs to be saved first). I don't want this behavior. I want to it work such that, the brightness of the image should change even without saving it first. The user should need to save it only once at the end.

3. What do you mean, "by shape objects" ? What is done?

4. So in if a new shape is drawn, the user can just move the shape around in the image to adjust its position. It is here: https://github.com/tzutalin/labelImg/blob/master/libs/canvas.py. (Line 164)

5. No, I mean move (like literally from one position to another)

[duddal_]

@mrjj
Is it fine if I start a new chat with you?

Thanks and Take care.

[mrjj]

@duddal_
Hi
You mean a new post or a chat ?
I dont mind chatting but it's not good for code so forums is actually better.

Ok we are talkling about the shapes.

I looked it over but i didn't see the brightness thing.

So about the moving thing. It seems they just adjust the 4 points for the rest.
At least that's what they do to move one pixel

self.selectedShape.points[0] += QPointF(-1.0, 0)

But not sure how this relates to the brightness

[duddal_]

@mrjj
Hello,

You will not find the brightness thing in the aforementioned link. I have added that separately in my code. And I cannot post it here and hence wanted to chat.

Yes exactly, that's how they do the movement on the shapes. I couldn't find a way of storing the image and the shapes drawn over the image in the memory temporarily and use them.

One more thing I thought of is mentioned here (opened a new issue as the problem is different):
https://forum.qt.io/topic/124498/how-to-remove-the-confirm-save-as-dialog-box-from-qfiledialog

What do you think?

[mrjj]

@duddal_

Ok. well chat is not good for code. The reason I ask about it is that I don't understand what you want to do with
The Shapes and this brightness change.

• . I couldn't find a way of storing the image and the shapes drawn over the image in the memory temporarily and use them.

well you can use them by looping over the list. Im not sure what you mean by storing them in memory but i aassume
something is done during saving and you want to do this without saving the image.
But Im not sure what exactly, you need help with.

you can move a shape like

            self.selectedShape.points[0] += QPointF(-1.0, 0)
            self.selectedShape.points[1] += QPointF(-1.0, 0)
            self.selectedShape.points[2] += QPointF(-1.0, 0)
            self.selectedShape.points[3] += QPointF(-1.0, 0)

i see no move function or similar

so you can loop over the shape list anbd move them by adjusting its points.
Is that what you mean by "move" ?

[duddal_]

@mrjj
Okay, let me frame it in another way.

Also, the "move" thing is only used as an example and is by no way related to our problem. This was just used as an example. Let's just forget about the "move" thing for now.

You said:

• I'm not sure what you mean by storing them in memory but I assume something is done during saving and you want to do this without saving the image.

And this is correct.

In order to annotate (draw shapes) the user needs to follow the following steps:

1. Load the directory containing images.
2. Draw the first shape and hit "Ctrl + S" (save) and save an XML file (with base-name same as image) that contains the name of the image, the coordinates of the shape (rectangle).
3. The user draws a few shapes over the same image and then has to hit the save button. After this, a dialog box pops up which asks the user to select the same (point 2) file and then asks for permission to overwrite this file. And the cycle repeats itself.

In order to avoid selecting the file to save again, I can set the self.defaultSaveDir as targetDirPath as mentioned here.

At first, I thought I have to save the shapes in the memory and then use the saved shapes in order to tackle the aforementioned tackle for a more concrete solution. But this is not the correct way I guess.

But the workaround works.

[duddal_]

@mrjj
How are you doing?

I am working on the same thing.

With labelImg, we can move the shapes by using the left mouse click and dragging the shape. However, I want to save the file as soon as the shape is moved. There is a function saveFile(True) in labelImg.py which helps us do exactly that.

After debugging the code, I found that there is a function called labelSelectionChanged(self) in lebalImg.py which is called every time the shape is selected and/or dragged. I inserted an if-statement with the condition being that if this function is called, save the file. However, I noticed that the file keeps saving in an infinity loop (i.e. did not stop, I had to stop the program by Ctrl+C).

One more solution I tried was: check if the Qt.ClosedHandCursor is used and then use that as a condition to save the file. This too did not work.

I would like some help with this. This seems very easy but I can't get my head around this.

[mrjj]

Hi
Welcome back.

Saving with every move might not work well as it might happen too fast so it's called many times.
I think that was what you saw when appeared to be in an infinity loop.

Do you really want to save to file each time user adjusts the shape just a little bit?

[duddal_]

Thanks for the reply!
So, the problem is as follows:

When the image is opened in the GUI, the program checks whether there is an annotation file (which contains shapes information) with the same name and then opens it with the image, thus, we can see the shapes with the image in the GUI. Now as you know that I have a button that changes the brightness even though new shapes are drawn. (We discussed this previously and the issue was just the save directory).

Now, when the shapes are loaded from the file and I move them and try to change the brightness by pushing a button, the program checks the annotation file in the directory and loads it. Therefore, the previous shapes are loaded again. I want it to keep the same shape (when the shape is moved). Therefore I thought, if I can save the shapes when they are moved, this might eliminate the problem. I mean, I don't know if there is another method with which this can be solved but that was just what I came up with.

[mrjj]

@duddal_

Hi
Well could you not just change code so even if you change brightness,
it won't reload the shapes?

Like adding a parameter to the function, so when called from the button,
it wont reload file but in all other case it will.

[duddal_]

Hi,
What do you mean exactly? Adding an argument to which function?

[mrjj]

@duddal_

Hi
I mean, it sounds odd why it would reload the files just because you adjust brightness so
i was wondering if you cant just change the code to make it not reload.

I don't know which functions, it was more a generic thought. :)

If you have a python debugger handy,
then it should not be that complex to find out why brightness change will reload the shapes
and make that not happen when its just brightness being adjusted.

[duddal_]

Okay Okay. I will make some changes to the code and probably paste it here so we can talk about it. Before that I have just one question, how can I check using an IF condition whether the shape was moved? (Perhaps comparing the coordinates from the previous point ?)

[mrjj]

@duddal_

Hi.
Ok. still not sure how you adjust brightness but maybe dont matter.

Well, the normal way to track if something moved
would be to take the difference between now and the last position.
So yes, that should work fine.