ASSERT: "isDetached()" with multithreaded QVectors

jars121 · 26 Apr 2018, 03:56

I have a number of QVectors in my application, each of which is modified (data inserted into a particular field of the QVector) by functions across a number of threads.

In debug() mode the application will run for a period of time (could be 5 seconds, could be a minute) before being killed with the following assertion:

ASSERT: "isDetached()" in file /opt/pathToinclude/include/QtCore/qvector.h, line 386
Aborted

The section of code in qvector.h resulting in the assertion on line 386 is as follows:

template <typename T>
void QVector<T>::detach()
{
    if (!isDetached()) {
#if !defined(QT_NO_UNSHARABLE_CONTAINERS)
        if (!d->alloc)
            d = Data::unsharableEmpty();
        else
#endif
            reallocData(d->size, int(d->alloc));
        }
        Q_ASSERT(isDetached());
    }

What exactly is happening here? Is the detachment occurring because simultaneous edits of a QVector from several threads are being attempted? If so, what is the recommended means for preventing this?

If not, what is going on here and what's the advise approaching for fixing it?

sierdzio · 26 Apr 2018, 04:00

@jars121 said in ASSERT: "isDetached()" with multithreaded QVectors:

What exactly is happening here? Is the detachment occurring because simultaneous edits of a QVector from several threads are being attempted? If so, what is the recommended means for preventing this?

Use a QMutex to guard the write.

Detaching is done when a copy of QVector is being done. That happens each time you pass a vector by value, or when you assign one vector to some other variable. Why it fails, though, I have no idea. Perhaps try asking on Qt interest mailing list where Qt devs reside - they'll surely know more details.

aha_1980 · replied to sierdzio on 26 Apr 2018, 04:00

@sierdzio said in ASSERT: "isDetached()" with multithreaded QVectors:

@jars121 said in ASSERT: "isDetached()" with multithreaded QVectors:

What exactly is happening here? Is the detachment occurring because simultaneous edits of a QVector from several threads are being attempted? If so, what is the recommended means for preventing this?

Use a QMutex to guard the write.

Detaching is done when a copy of QVector is being done. That happens each time you pass a vector by value, or when you assign one vector to some other variable. Why it fails, though, I have no idea.

if you modify the object from two threads, anything can happen.

i assume the vector is already in the state of detaching and then requested to detach again from the second thread.

jars121 · replied to sierdzio on 26 Apr 2018, 04:22

@sierdzio said in ASSERT: "isDetached()" with multithreaded QVectors:

@jars121 said in ASSERT: "isDetached()" with multithreaded QVectors:

What exactly is happening here? Is the detachment occurring because simultaneous edits of a QVector from several threads are being attempted? If so, what is the recommended means for preventing this?

Use a QMutex to guard the write.

Detaching is done when a copy of QVector is being done. That happens each time you pass a vector by value, or when you assign one vector to some other variable. Why it fails, though, I have no idea. Perhaps try asking on Qt interest mailing list where Qt devs reside - they'll surely know more details.

Thank you @sierdzio. I'll post on the mailing list as recommended, as I've been making changes to my code all day and don't appear to be making any progress.

@aha_1980 said in ASSERT: "isDetached()" with multithreaded QVectors:

@sierdzio said in ASSERT: "isDetached()" with multithreaded QVectors:

@jars121 said in ASSERT: "isDetached()" with multithreaded QVectors:

What exactly is happening here? Is the detachment occurring because simultaneous edits of a QVector from several threads are being attempted? If so, what is the recommended means for preventing this?

Use a QMutex to guard the write.

Detaching is done when a copy of QVector is being done. That happens each time you pass a vector by value, or when you assign one vector to some other variable. Why it fails, though, I have no idea.

if you modify the object from two threads, anything can happen.

i assume the vector is already in the state of detaching and then requested to detach again from the second thread.

Thank you @aha_1980. Would sierdzo's recommendation of using a QMutex aid in preventing this in any way? If not, can you recommend a strategy? Would replace QVectors with dedicated variables solve this issue?

I.e. I'm using a QVector to save the output values for 5 calculations. I.e. outputVector=[v1, v2, v3, v4, v5]. v1 through v5 can be updated from different threads, which is likely the cause of the issue. If, instead of using outputVector, I had outputValue1, outputValue2, ..., outputValue5, this issue should be fixed?

If I did use this approach, is there a way to loop through the variables in a for loop like I could with a QVector?

e.g.

for (int i(i); i < 6; i++) {
    outputValue[i] = 5;
}

The above is possible (although not written in that form obviously) in Python, but I haven't come across it in c++ as of yet. I'd happily swap my current QVectors with dedicated variables if the ability to address the variable within a for loop were possible.

aha_1980 · wrote on 26 Apr 2018, 06:35

@jars121 said in ASSERT: "isDetached()" with multithreaded QVectors:

Would sierdzo's recommendation of using a QMutex aid in preventing this in any way? If not, can you recommend a strategy? Would replace QVectors with dedicated variables solve this issue?

Using mutexes or semaphores is the usual way of protecting variables against concurrent access.

The only variables that don't need protection are variables that can be changed in one (atomic) CPU cycle. This depends a bit on the machine and needs a deep understanding whats going on from programmes side.

Also note, that simple operations in C decode to multiple machine commands: int a = 3; usually means: load the constant 3 into a register and then store it into the memory occupied by a.

Even worse ist the following: a += 4 because that means: load the value from the memory a into a register, add the constant 4 and store the value back. This is not atomic. If you do this from two threads, a can contain anything afterwards.

Regards

jars121 · wrote on 26 Apr 2018, 22:43

A bit of a development; I added a qDebug() << "Main Thread: "" << QThread::currentThread(); at the start of main.cpp, and it appears that my main class is being run twice. This is definitely not per the design, and is likely causing all sorts of issues. I'm tracing through the code at the moment and will hopefully fix this issue when I can resolve the doubling up.

jars121 · 27 Apr 2018, 04:52

I appear to have fixed the issue with the following changes:

Removed the duplication error mentioned above; I was accidentally instantiating my main class twice.
I created a new class to manage the shared QVectors separate to my main class.
I've incorporated QMutexLockers into each function which modifies the shared QVectors.

The application has been running for around 20 minutes now without issue, so I'm hopeful that the detaching error is resolved.

aha_1980 · replied to jars121 on 27 Apr 2018, 04:52

@jars121 thanks for your feedback.

sounds like a clean solution.

so please mark this topic as SOLVED now. Thanks

SGaist · wrote on 27 Apr 2018, 06:26

Hi,

Just an additional tool worth mentioning: QReadWriteLock. If you are reading from several threads then you can easily access your data and as soon as there's a write going it will lock the access. It might be faster than QMutex.

jars121 · wrote on 27 Apr 2018, 11:33

Thanks @SGaist I had seen mention of QReadWriteLock in my travels, but hadn't yet put any serious thought into it. I'll be sure to read further and do some testing between it and QMutex.

jars121 · wrote on 11 May 2018, 02:55

I've marked this as 'Unsolved', as I'm afraid I'm still battling this "isDetached()" assertion.

Following SGaist's recommendation above, I've actually gone through my entire application and removed all QMutex, and replaced them with QReadWriteLocks. In short, wherever I'm reading from or writing to a shared (between QThreads) QVector, QList, etc., there's a QReadLocker or QWriteLocker (respectively) present. However, the issue remains that after a seemingly random period of time, the application will crash (in Debug mode) with the "isDetached()" assertion.

I've done some further research on this issue, and found a similar reference to this assertion here. The findings/recommendations in that thread boil down to this:

Using the [] operator to retrieve an item at the specified index in a QVector can create a deep copy situation, which is from where a detached() scenario could arise. This is specified in the QVector Class documentation.
Using at() to retrieve an item at the specified index in a QVector is the preferred approach if not modifying the QVector.

As it turns out, I've never used at() before, and have been completely unaware of this distinction. I'm going through my code now, with the intention of replacing all read-only QVector operator references with at().

The use of at() won't assert "isDetached()", so hopefully this solves this particular issue once and for all. I'll update the thread with my findings.

jars121 · wrote on 11 May 2018, 05:36

Well the above certainly doesn't appear to have made any difference whatsoever. I've replaced every single instance of read-only [] QVector and QList operator with .at(), but the "isDetached()" assertion is still present.

One question as I continue to review the c++ side of my application; when reading from an exposed c++ QVector in QML, is vector[i] the only/recommended way of retrieving a value from index i? The reason I ask is I might accidentally be inducing a deep copy on the QML side without knowing it. The documentation suggests that the [] operators are the correct approach, but I wanted to confirm before dismissing QML as the potential cause of this extremely frustrating issue.

Christian Ehrlicher · wrote on 11 May 2018, 05:40

This will definitely not fix your problems but only hide it. When using objects in multiple threads you have to sync the access with a mutex or similar as explained by SGaist and others. If you don't do it you will get into trouble sooner or later.

jars121 · wrote on 11 May 2018, 06:00

I figured that's what I was doing by using QReadWriteLock and corresponding QReadLockers and QWriteLockers when interacting with the shared QVectors, QLists, etc?

jars121 · 20 Jul 2018, 21:15

Ok so I've removed all the QReadWriteLock, QReadLocker and QWriteLocker instances, and have gone back to QMutex with QMutexLockers. I've been somewhat more diligent this time around, and in combination with the change from [] operators to .at() the application seems to be running smoothly.

I've got further testing to do before I'm completely comfortable, so I'll leave the thread open for the time being.

Yippiyak · replied to jars121 on 20 Jul 2018, 21:15

@jars121

did you ever solve this issue? I have the same problem

Zhengyang · 2 Aug 2021, 17:53

Hi, @jars121
Have you found a solution? I have the same problem. And I can not solve it by adding a QMutex.

SGaist · replied to Zhengyang on 2 Aug 2021, 17:53

@Zhengyang hi and welcome to devnet, it's described in the last post of @jars121.

If QMutex does not solve your issue, you are likely misusing it.