changing from 1:1 to 1:many

mzimmers

Hi all -

As many of you know, I've been developing an app that allows a user to configure and monitor a remote device. When this project began, it was anticipated that my app would be used to deal with one device at a time.

Now, sales is asking for "mass processing" features. In other words, from a list of devices (which I already have), select some and push a button, and something (like a firmware OTA update) happens to all the selected devices.

My app wasn't architected for this, and I'm wondering how best to go about it. My first thought was to create new threads for each selected device, and write wrappers for whatever functionality I need to "mass." This could get messy...I welcome any ideas on this.

Thanks...

SGaist

Hi,

How are you communicating with your devices ?

mzimmers

@SGaist oh yeah, I guess that would be helpful: UDP sockets, usually multicast. I think I can make things work with a common communications layer; it's just the "work" work that might need to be divvied up.

SGaist

Are you sure UDP is the right tool to push firmware updates ?

mzimmers

I misspoke...we use UDP for other information, but we use TCP for the OTA. It goes through some rudimentary encryption as well.

SGaist

Ok, then it sounds like you could have some sort of task manager. Maybe QtConcurrent could the job depending on what these tasks are.

kshegunov

@mzimmers said in changing from 1:1 to 1:many:

My app wasn't architected for this, and I'm wondering how best to go about it. My first thought was to create new threads for each selected device, and write wrappers for whatever functionality I need to "mass." This could get messy...I welcome any ideas on this.

As each job sounds to be completely independent of each other piece of work, what @SGaist says sounds most sensible to me - putting the work into a queue and spreading it over a thread pool. Implementations can vary depending on the required complexity and the specifics.

Creating one thread for each piece is borderline terrible. Threads can be expensive, and you don't need that many anyway. You could even just run one thread that does all the work, which'd be my first go-to.

mzimmers

@kshegunov so I've read a little of the documentation on QtConcurrent. If I get the gist of it, I'd create a new class, RemoteDevice, which would be the basis of my sequence. Each element in my sequence would represent one (selected) remote device. The function specified in the mapping call would be something that receives signals from the worker thread, and acts on them. Is this even close to being correct?

Thanks...

kshegunov

@mzimmers said in changing from 1:1 to 1:many:

@kshegunov so I've read a little of the documentation on QtConcurrent. If I get the gist of it, I'd create a new class, RemoteDevice, which would be the basis of my sequence. Each element in my sequence would represent one (selected) remote device. The function specified in the mapping call would be something that receives signals from the worker thread, and acts on them. Is this even close to being correct?

Hm, I don't understand your explanation, so I can't say if it's correct. It can go as simple as this:

class PushToDevice
{
public:
    PushToDevice(...); //< Save some arguments you may need when it executes
    
    void operator () ()
    {
        // Do whatever is you do in the thread
    }
}

QtConcurrent::run(PushToDevice(...));

Alternatively:

class PushToDevice : public QRunnable
{
    // ... Save arguments through the constructor or w/e else you need
    void run()
    {
         // Do whatever it is in the thread
    }
};

QThreadPool * pool = QThreadPool::globalInstance();
pool->start(new PushToDevice(...));

Something along these lines is what I was thinking about.

Note: Opening sockets or db connections should happen from within the thread they're intended to be used

mzimmers

@kshegunov I'm intrigued by the QtConcurrent namespace approach. A few questions/comments:

I've never seen the "void operator () ()" construct before -- I read a little about it, but I'm not sure I fully understand it. What is its advantage over a traditional function?
in your first example, where is the PushToDevice object instantiated -- does QtConcurrent:::run() do this for you?
I take it that QtConcurrent::run() would typically be run from a worker thread, yes?
Your comment about opening sockets in the thread that will use them is interesting. Is this a Qt-specific guideline? I ask because I wrote the firmware on my target device specifically NOT to do this. (FreeRTOS doesn't have threads, but it does have tasks.) I have a socket task that manages the state of all the sockets. Access to these sockets is through queue messages. This way I can provide a common interface to the socket layer for all the other tasks.

kshegunov

@mzimmers said in changing from 1:1 to 1:many:

I've never seen the "void operator () ()" construct before -- I read a little about it, but I'm not sure I fully understand it. What is its advantage over a traditional function?

It's the old way of writing a lambda (a.k.a. functor object). The difference from a regular function is that you have a state attached to the actual function.

in your first example, where is the PushToDevice object instantiated -- does QtConcurrent:::run() do this for you?

No, you pass it to QtConcurrent::run, I've used an rvalue (a temporary) here just for brevity and've passed it by value.

I take it that QtConcurrent::run() would typically be run from a worker thread, yes?

When the time comes, yes. It's queued and processed when a timeslice becomes available in the thread pool.

Is this a Qt-specific guideline?

Yes, a Qt-specific requirement and I didn't mean sockets as unix descriptors, but sockets as QTcpSocket and such.

I ask because I wrote the firmware on my target device specifically NOT to do this.

Can't comment. I've not worked with RTOSes, but Qt's sockets can be somewhat finicky when you need to get threads involved. One such example is when you develop a threaded server. What one has to do (as described in the docs) is to transfer the descriptor to the correct thread and then initialize the socket object. And if a socket is in a specific thread it's a bit different than the regular QObject as it can't be freely pushed into another thread. A rather inconsistent behaviour compared to other (regular) QObjects, but that's an implementation limitation unfortunately.

mzimmers

Thanks for the suggestions. I'm sure I'll have more questions about implementation, but as the original issue has been answered, I'm marking this as solved.