Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Modify QVariant data directly



  • Hi,

    I'm working on some classes and have a (common) problem with QVariant.

    I want to work with different types at the same time and would like to avoid using some base class with inheritance since i would need wrappers for everything. So I thought about using QVariant.

    The problem is that my classes are intended to store relatively big data elements. Thus, copying them all the time would be a problem, not to mention i would have to pass them pack to my structure manually every time.

    Also, it would be bad because I need to cache my data since files may grow as big as several gb.

    I didn't find anything about how to access the Data stored in QVariant directly, all posts regarding this say you have to copy, so i took a look at the source code.

    QVariant uses a d-pointer which is not private, thus you can inherit from QVariant and add methods to get to the data without copying.

    I looked at

    QVariant::value<T>
    

    first. it calls

    qvariant_cast<T>(*this)
    

    which calls

    QtPrivate::QVariantValueHelperInterface<T>::invoke(v);
    

    QtPrivate is just a namespace. QVariantValueHelperInterface has a few specializations with an invoke method. but QVariantValueHelperInterface<T> doesn't. It inherits from QVariantValueHelper, but that class doesn't have that method too. It should be

    static T invoke(const QVariant &v)
    

    So I looked around some more and found something. Here is a quick draft

    class QxVariant : public QVariant
    {
        template<typename T>
        struct QVariantIntegrator
        {
            static const bool CanUseInternalSpace = sizeof(T) <= sizeof(QVariant::Private::Data)
                    && ((QTypeInfoQuery<T>::isRelocatable) || std::is_enum<T>::value);
            typedef std::integral_constant<bool, CanUseInternalSpace> CanUseInternalSpace_t;
        };
        Q_STATIC_ASSERT(QVariantIntegrator<double>::CanUseInternalSpace);
        Q_STATIC_ASSERT(QVariantIntegrator<long int>::CanUseInternalSpace);
        Q_STATIC_ASSERT(QVariantIntegrator<qulonglong>::CanUseInternalSpace);
    
        #ifdef Q_CC_SUN // Sun CC picks the wrong overload, so introduce awful hack
    
        // takes a type, returns the internal void* pointer cast
        // to a pointer of the input type
        template <typename T>
        static inline T *v_cast(const QVariant::Private *nd, T * = 0)
        {
            QVariant::Private *d = const_cast<QVariant::Private *>(nd);
            return !QVariantIntegrator<T>::CanUseInternalSpace
                    ? static_cast<T *>(d->data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&d->data.c));
        }
    
        #else // every other compiler in this world
    
        template <typename T>
        static inline const T *v_cast(const QVariant::Private *d, T * = 0)
        {
            return !QVariantIntegrator<T>::CanUseInternalSpace
                    ? static_cast<const T *>(d->data.shared->ptr)
                    : static_cast<const T *>(static_cast<const void *>(&d->data.c));
        }
    
        template <typename T>
        static inline T *v_cast(QVariant::Private *d, T * = 0)
        {
            return !QVariantIntegrator<T>::CanUseInternalSpace
                    ? static_cast<T *>(d->data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&d->data.c));
        }
    
        #endif
    
    public:
        QxVariant(){}
        template <typename T>
        QxVariant(T t) : QVariant(t) {}
    
    
        template <typename T>
        T* pointer()
        {
            QVariant::Private x = (this->d);
            return v_cast<T>(&x);
        }
        template <typename T>
        const T* const_pointer()
        {
            const QVariant::Private x = (this->d);
            return v_cast<T>(&x);
        }
    
        template <typename T>
        T & reference()
        {
            QVariant::Private x = (this->d);
            return *(v_cast<T>(&x));
        }
        template <typename T>
        const T & const_reference()
        {
            const QVariant::Private x = (this->d);
            return *(v_cast<T>(&x));
        }
    
    };
    

    The private part is a copy from qvariant_p.h. I just made v_cast static in here. The rest is just a quick test to see if it works.

    Getting a pointer and a const pointer works for Objects (I tested with QRect), but not with e.g. int.

    The references seem to work for reading the values, but return a copy instead of a reference.

    I hope I can get some help with this here.

    ps: does someone know what the T*=0 in v_cast(const QVariant::Private *d, T * = 0) is good for? It isn't used even once from what i can tell.

    pps: Is there a reason why there is no such method in QVariant in the first place?



  • update:

    I did some more digging and it seems like i have overseen a few definitions, making things a little easier:

    class QxVariant : public QVariant
    {
    
    public:
        QxVariant(){}
        template <typename T>
        QxVariant(T t) : QVariant(t) {}
    
        template <typename T>
        T* pointer()
        {
            return (this->d.is_shared) 
                    ? static_cast<T *>(this->d.data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
        
        template <typename T>
        T* const_pointer()
        {
            return (this->d.is_shared) 
                    ? static_cast<T *>(this->d.data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
        
        template <typename T>
        T & ref()
        {
            return (this->d.is_shared) 
                    ? * static_cast<T *>(this->d.data.shared->ptr)
                    : * static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
        
        template <typename T>
        T & const_ref()
        {
            return (this->d.is_shared) 
                    ? * static_cast<T *>(this->d.data.shared->ptr)
                    : * static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
    
    };
    

    But the behaviour is the same as before. But i found out that the pointer works for QVariant v(T &t), but not with v.fromValue(t);

    Btw, if you want to get the data, QVariant actually defines

    public:
        typedef Private DataPtr;
        inline DataPtr &data_ptr() { return d; }
        inline const DataPtr &data_ptr() const { return d; }
    

    to get it, but it's not documented.



  • Another update:

    The content of QVariant is inside a union. That union is d.data. It has, aside from the usual char int etc., a member shared, which itself holds a void*. So if d.is_shared is true, the data isn't actually inside the union, not even as pointer.

    If i create the QVariant using QVariant (T &t), is_shared is true, and my pointer function seems to work (for Objects like QRect).

    If it's a primitive like int or set by fromValue(t), is_shared is false and thus it's inside the union. I don't have much experience with unions, but from what i read i can't return a reference to a member of a union since the type is unknown. That might explain why i get a copy in some cases instead of a reference.


  • Lifetime Qt Champion

    Hi,

    What about using QSharedData and its friends to make your data implicitly shared like QByteArray, QImage, etc. ?



  • Hi,
    thanks for your reply. As i understand it, QSharedData relies on inheriting a common class/interface, which is something i would like to avoid since i want to use existing classes without wrappers.

    But i think i found the solution with QVariant myself. I will test it and if everything works fine post it here for others who would like to use it.



  • Hi

    here is my solution. Was much easier than i thought, kept looking at the wrong places.

    class QxVariant : public QVariant
    {
    
    public:
        QxVariant(){}
        template <typename T>
        QxVariant(T &t) : QVariant(t) {}
        
        template <typename T>
        T* pointer()
        {
            return (this->d.is_shared)
                    ? static_cast<T *>(this->d.data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
    
        template <typename T>
        const T* const_pointer()
        {
            return (this->d.is_shared)
                    ? static_cast<T *>(this->d.data.shared->ptr)
                    : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
        }
    
        template <typename T>
        T & ref()
        {
            static T *tRef;
            tRef = pointer<T>();
            return *tRef;
        }
    
        template <typename T>
        const T & const_ref()
        {
            static T *tRef;
            tRef = pointer<T>();
            return *tRef;
        }
        
    };
    
    template <typename T>
    class QxVariantPointer
    {
      QxVariant* m_variant;
      
    public:
      QxVariantPointer(QVariant *v) : m_variant(static_cast<QxVariant*>(v)){}
      QxVariantPointer(QxVariant *v) : m_variant(v){}
      
      T* operator ->()
      {
          return m_variant->pointer<T>();
      }
      
      //other operator overloads
      
    };
    

    The smart pointer is for a more native usage, but didn't want post more operators (too long).

    Fyi this doesn't even use elements from a private header, only the public one. Thus, this should work with all versions.

    I hope some people will find this helpfull or have some improvements.



  • Had some time too look into QSharedData. When I first read it (too shallow) and answered, I had misunderstood it's purpose.

    I can say now that it's not what I needed. My problem wasn't about avoiding copying for read only operations and thus can't be prevented using copy on write.

    My problem was about implementing a container like class that holds various types, e.g. like QList<QVariant>, but without copying the data because my container needs to stay the owner to load and unload elements for caching and other stuff.

    That being said, I will definitely take a closer look at QSharedData. I knew about Qt's d-pointers, but didn't know they even provide classes for implementing it. I'm sure I will find use for it sooner than later, so thanks for that information.

    One last question. When I use QSharedData and friends in my class and I want to serialize that class, I just need to define serialization for QSharedDataPointer and the data class, just like normally with other classes?


  • Lifetime Qt Champion

    It'an implementation detail, you should do it like a normal class.


Log in to reply