Modify QVariant data directly

[Larvae]

Hi,

I'm working on some classes and have a (common) problem with QVariant.

I want to work with different types at the same time and would like to avoid using some base class with inheritance since i would need wrappers for everything. So I thought about using QVariant.

The problem is that my classes are intended to store relatively big data elements. Thus, copying them all the time would be a problem, not to mention i would have to pass them pack to my structure manually every time.

Also, it would be bad because I need to cache my data since files may grow as big as several gb.

I didn't find anything about how to access the Data stored in QVariant directly, all posts regarding this say you have to copy, so i took a look at the source code.

QVariant uses a d-pointer which is not private, thus you can inherit from QVariant and add methods to get to the data without copying.

I looked at

QVariant::value<T>

first. it calls

qvariant_cast<T>(*this)

which calls

QtPrivate::QVariantValueHelperInterface<T>::invoke(v);

QtPrivate is just a namespace. QVariantValueHelperInterface has a few specializations with an invoke method. but QVariantValueHelperInterface<T> doesn't. It inherits from QVariantValueHelper, but that class doesn't have that method too. It should be

static T invoke(const QVariant &v)

So I looked around some more and found something. Here is a quick draft

class QxVariant : public QVariant
{
    template<typename T>
    struct QVariantIntegrator
    {
        static const bool CanUseInternalSpace = sizeof(T) <= sizeof(QVariant::Private::Data)
                && ((QTypeInfoQuery<T>::isRelocatable) || std::is_enum<T>::value);
        typedef std::integral_constant<bool, CanUseInternalSpace> CanUseInternalSpace_t;
    };
    Q_STATIC_ASSERT(QVariantIntegrator<double>::CanUseInternalSpace);
    Q_STATIC_ASSERT(QVariantIntegrator<long int>::CanUseInternalSpace);
    Q_STATIC_ASSERT(QVariantIntegrator<qulonglong>::CanUseInternalSpace);

    #ifdef Q_CC_SUN // Sun CC picks the wrong overload, so introduce awful hack

    // takes a type, returns the internal void* pointer cast
    // to a pointer of the input type
    template <typename T>
    static inline T *v_cast(const QVariant::Private *nd, T * = 0)
    {
        QVariant::Private *d = const_cast<QVariant::Private *>(nd);
        return !QVariantIntegrator<T>::CanUseInternalSpace
                ? static_cast<T *>(d->data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&d->data.c));
    }

    #else // every other compiler in this world

    template <typename T>
    static inline const T *v_cast(const QVariant::Private *d, T * = 0)
    {
        return !QVariantIntegrator<T>::CanUseInternalSpace
                ? static_cast<const T *>(d->data.shared->ptr)
                : static_cast<const T *>(static_cast<const void *>(&d->data.c));
    }

    template <typename T>
    static inline T *v_cast(QVariant::Private *d, T * = 0)
    {
        return !QVariantIntegrator<T>::CanUseInternalSpace
                ? static_cast<T *>(d->data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&d->data.c));
    }

    #endif

public:
    QxVariant(){}
    template <typename T>
    QxVariant(T t) : QVariant(t) {}


    template <typename T>
    T* pointer()
    {
        QVariant::Private x = (this->d);
        return v_cast<T>(&x);
    }
    template <typename T>
    const T* const_pointer()
    {
        const QVariant::Private x = (this->d);
        return v_cast<T>(&x);
    }

    template <typename T>
    T & reference()
    {
        QVariant::Private x = (this->d);
        return *(v_cast<T>(&x));
    }
    template <typename T>
    const T & const_reference()
    {
        const QVariant::Private x = (this->d);
        return *(v_cast<T>(&x));
    }

};

The private part is a copy from qvariant_p.h. I just made v_cast static in here. The rest is just a quick test to see if it works.

Getting a pointer and a const pointer works for Objects (I tested with QRect), but not with e.g. int.

The references seem to work for reading the values, but return a copy instead of a reference.

I hope I can get some help with this here.

ps: does someone know what the T*=0 in v_cast(const QVariant::Private *d, T * = 0) is good for? It isn't used even once from what i can tell.

pps: Is there a reason why there is no such method in QVariant in the first place?

[Larvae]

update:

I did some more digging and it seems like i have overseen a few definitions, making things a little easier:

class QxVariant : public QVariant
{

public:
    QxVariant(){}
    template <typename T>
    QxVariant(T t) : QVariant(t) {}

    template <typename T>
    T* pointer()
    {
        return (this->d.is_shared) 
                ? static_cast<T *>(this->d.data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }
    
    template <typename T>
    T* const_pointer()
    {
        return (this->d.is_shared) 
                ? static_cast<T *>(this->d.data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }
    
    template <typename T>
    T & ref()
    {
        return (this->d.is_shared) 
                ? * static_cast<T *>(this->d.data.shared->ptr)
                : * static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }
    
    template <typename T>
    T & const_ref()
    {
        return (this->d.is_shared) 
                ? * static_cast<T *>(this->d.data.shared->ptr)
                : * static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }

};

But the behaviour is the same as before. But i found out that the pointer works for QVariant v(T &t), but not with v.fromValue(t);

Btw, if you want to get the data, QVariant actually defines

public:
    typedef Private DataPtr;
    inline DataPtr &data_ptr() { return d; }
    inline const DataPtr &data_ptr() const { return d; }

to get it, but it's not documented.

[Larvae]

Another update:

The content of QVariant is inside a union. That union is d.data. It has, aside from the usual char int etc., a member shared, which itself holds a void*. So if d.is_shared is true, the data isn't actually inside the union, not even as pointer.

If i create the QVariant using QVariant (T &t), is_shared is true, and my pointer function seems to work (for Objects like QRect).

If it's a primitive like int or set by fromValue(t), is_shared is false and thus it's inside the union. I don't have much experience with unions, but from what i read i can't return a reference to a member of a union since the type is unknown. That might explain why i get a copy in some cases instead of a reference.

[SGaist]

Hi,

What about using QSharedData and its friends to make your data implicitly shared like QByteArray, QImage, etc. ?

[Larvae]

Hi,
thanks for your reply. As i understand it, QSharedData relies on inheriting a common class/interface, which is something i would like to avoid since i want to use existing classes without wrappers.

But i think i found the solution with QVariant myself. I will test it and if everything works fine post it here for others who would like to use it.

[Larvae]

Hi

here is my solution. Was much easier than i thought, kept looking at the wrong places.

class QxVariant : public QVariant
{

public:
    QxVariant(){}
    template <typename T>
    QxVariant(T &t) : QVariant(t) {}
    
    template <typename T>
    T* pointer()
    {
        return (this->d.is_shared)
                ? static_cast<T *>(this->d.data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }

    template <typename T>
    const T* const_pointer()
    {
        return (this->d.is_shared)
                ? static_cast<T *>(this->d.data.shared->ptr)
                : static_cast<T *>(static_cast<void *>(&(this->d.data.c)));
    }

    template <typename T>
    T & ref()
    {
        static T *tRef;
        tRef = pointer<T>();
        return *tRef;
    }

    template <typename T>
    const T & const_ref()
    {
        static T *tRef;
        tRef = pointer<T>();
        return *tRef;
    }
    
};

template <typename T>
class QxVariantPointer
{
  QxVariant* m_variant;
  
public:
  QxVariantPointer(QVariant *v) : m_variant(static_cast<QxVariant*>(v)){}
  QxVariantPointer(QxVariant *v) : m_variant(v){}
  
  T* operator ->()
  {
      return m_variant->pointer<T>();
  }
  
  //other operator overloads
  
};

The smart pointer is for a more native usage, but didn't want post more operators (too long).

Fyi this doesn't even use elements from a private header, only the public one. Thus, this should work with all versions.

I hope some people will find this helpfull or have some improvements.

[Larvae]

Had some time too look into QSharedData. When I first read it (too shallow) and answered, I had misunderstood it's purpose.

I can say now that it's not what I needed. My problem wasn't about avoiding copying for read only operations and thus can't be prevented using copy on write.

My problem was about implementing a container like class that holds various types, e.g. like QList<QVariant>, but without copying the data because my container needs to stay the owner to load and unload elements for caching and other stuff.

That being said, I will definitely take a closer look at QSharedData. I knew about Qt's d-pointers, but didn't know they even provide classes for implementing it. I'm sure I will find use for it sooner than later, so thanks for that information.

One last question. When I use QSharedData and friends in my class and I want to serialize that class, I just need to define serialization for QSharedDataPointer and the data class, just like normally with other classes?

[SGaist]

It'an implementation detail, you should do it like a normal class.