crazy frameworks integration.... and I'm stuck



  • Hi!

    Here is what I'm implementing at the moment:

    • Hardware Environment:
      A custom USB device is connected as device to Android phone, exposing its USB port as OTG one.

    • My goal:
      Gui app, reading USB device under the hood in separate thread and displaying streamed data in QML widgets.
      I could proceed without QML, relaying solely on Qt's C++ GUI, but often gui code refactoring makes me think, I'd be better stick to QML...

    So far, basic integration Qt C++ <-> NDK <-> Java's UsbManager works fine. Device is feeding data correctly into QRunnable, started by QthreadPool. It happens in a native Java's method, called from within Java's environment.

    My Trouble: I can't manage my separate thread to stop reading device and exit when Back button is pressed....
    Indeed, when I press it, GUI just freezes and separate thread continues handling incoming data.

    Any clues how to interconnect QML stuff, started from main.c...

    QQmlApplicationEngine engine;
    engine.load(QUrl("qrc:/MainWindow.qml"));
    

    inside a JNI native call?

    JNIEXPORT void JNICALL Java_com_example_code_MyActivity_notifyDeviceAttached
      (JNIEnv *, jclass, jint fd)
    {
        glue = new Glue(fd); // fd - is usb device descriptor opened from Java's part of code.
        glue->setAutoDelete(false);
    
        QThreadPool *threadPool = QThreadPool::globalInstance();
        threadPool->start(glue);
    
    }
    
    ....
    
    void Glue::run()
    {
        while(running)
        {
            struct usbdevfs_bulktransfer data;
            uint8_t buf[64] = {0};
            data.ep = 0x82;
            data.len = 64;
            data.data = buf;
            data.timeout = 1000;
            int res = ioctl(fd, USBDEVFS_BULK, &data);
            qDebug() <<  res << buf[0] << buf[1] << buf[2] << buf[3] << buf[4] << buf[5] << buf[6] << buf[7];
        }
    
    }
    

    by the way, if I unplug my device from Android, this call makes the app normally quit:

    JNIEXPORT void JNICALL Java_com_example_code_MyActivity_notifyDeviceDetached
      (JNIEnv *, jclass, jint fd)
    {
        glue->setAutoDelete(true);
        glue->stop();
        QCoreApplication *qapp = QCoreApplication::instance();
        qapp->quit();
    }
    


  • issue is about the while loop which is continuously running. This thread is not exiting. Since you said UI freezes, it could be that main thread is actually doing some time consuming task. How about subclassing the QThread and using the timers to read the data every few milliseconds & exit the thread when you hit the back button ?



  • @dheerendra Hi! Thank you for a clue! I think you're right, pointing the loop. I really don't know the easy method to catch Back Button in Qt. What I could perform, is just kill the app from Java's side:

    public class MyActivity extends QtActivity
    {
    ...
        @Override
        public boolean onKeyDown(int keyCode, KeyEvent event) {
            if ((keyCode == KeyEvent.KEYCODE_BACK)) {
                finish();
                System.exit(0);
            }
            return true;
        //    super.injectEvent(event);
        }
    
    


  • ok, just to mark the topic resolved...

    Finally, I have managed all the parts of the story to talk to each other... seems to work, but may be something is not perfect, so I'll appreciate all comments/criticism/etc.

    I'll try to explain it in a howto manner, so excuse the long story.

    1. add QT += androidextras to QT's .pro file

    2. add device_filter.xml to <YOUR PROJECT ROOT>/android/res/xml directory:

    <?xml version="1.0" encoding="utf-8"?>
    <resources>
        <usb-device vendor-id="123" product-id="123"/>
    </resources>
    
    

    123, 123 - vendorId and productId of your USB device

    1. create MyActivity.java in <YOUR PROJECT ROOT>/android/src/com/mycompany/myapp dirertory (I found the source in some stackoverflow's answers and ammended it slightly...)
    package com.mycompany.myapp;
    import org.qtproject.qt5.android.bindings.QtActivity;
    import android.os.Bundle;
    import android.util.Log;
    import android.app.PendingIntent;
    import android.content.BroadcastReceiver;
    import android.content.Context;
    import android.content.Intent;
    import android.content.IntentFilter;
    import android.hardware.usb.UsbConstants;
    import android.hardware.usb.UsbDevice;
    import android.hardware.usb.UsbDeviceConnection;
    import android.hardware.usb.UsbEndpoint;
    import android.hardware.usb.UsbInterface;
    import android.hardware.usb.UsbAccessory;
    import android.hardware.usb.UsbManager;
    import android.os.Bundle;
    import android.os.Handler;
    import android.text.Layout;
    import android.util.Log;
    import android.app.Activity;
    import android.view.KeyEvent;
    
    import org.xmlpull.v1.XmlPullParser;
    import org.xmlpull.v1.XmlPullParserException;
    import org.xmlpull.v1.XmlPullParserFactory;
    
    import java.util.Arrays;
    import java.util.HashMap;
    import java.util.Iterator;
    import java.lang.Runnable;
    import java.util.concurrent.locks.ReentrantLock;
    
    import static java.lang.Integer.parseInt;
    import static java.sql.Types.NULL;
    import java.io.IOException;
    import java.lang.Byte;
    
    //android:name="org.qtproject.qt5.android.bindings.QtActivity"
    
    public class MyActivity extends QtActivity
    {
        private static MyActivity m_instance;
        private UsbAccessory accessory;
        private String TAG = "dfulog";
        private static final String ACTION_USB_PERMISSION = "com.mycompany.myapp.USB_PERMISSION";
        private PendingIntent mPermissionIntent;
        private UsbManager manager;
        private UsbDeviceConnection connection;
        private HashMap<Integer, Integer> connectedDevices;
        private int vendorId = 0, productId = 0;
    
    
        @Override
        public boolean onKeyDown(int keyCode, KeyEvent event) {
            if ((keyCode == KeyEvent.KEYCODE_BACK)) {
                finish();
                System.exit(0);
            }
            return true;
        //    super.injectEvent(event);
        }
    
    
        public MyActivity()
        {
            m_instance = this;
            connectedDevices = new HashMap<Integer, Integer>();
    
    
        }
    
        @Override
        public void onCreate(Bundle savedInstanceState)
        {
            super.onCreate(savedInstanceState);
            XmlPullParserFactory factory;
            try {
                factory = XmlPullParserFactory.newInstance();
                factory.setNamespaceAware(true);
                //XmlResourceParser xrp = context.getResources().getXml(R.xml.encounters);
                XmlPullParser xpp = getResources().getXml(R.xml.device_filter);
    
                int eventType = xpp.getEventType();
                while (eventType != XmlPullParser.END_DOCUMENT) {
    
                    if (eventType == XmlPullParser.START_TAG && xpp.getName().equals("usb-device")) {
                        vendorId = parseInt(xpp.getAttributeValue(null, "vendor-id"));
                        productId = parseInt(xpp.getAttributeValue(null, "product-id"));
                    }
                    eventType = xpp.next();
                }
            } catch (XmlPullParserException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
    
            manager = (UsbManager) getSystemService(Context.USB_SERVICE);
    
            registerReceiver(usbManagerBroadcastReceiver, new IntentFilter(UsbManager.ACTION_USB_DEVICE_ATTACHED));
            registerReceiver(usbManagerBroadcastReceiver, new IntentFilter(UsbManager.ACTION_USB_DEVICE_DETACHED));
            registerReceiver(usbManagerBroadcastReceiver, new IntentFilter(ACTION_USB_PERMISSION));
    
            mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
    
            final Handler handler = new Handler();
    
            handler.postDelayed(new Runnable()
            {
                @Override
                public void run()
                {
                    checkForDevices();
                }
            }, 1000);
        }
    
        @Override
        public void onDestroy()
        {
            super.onDestroy();
        }
    
        @Override
        protected void onActivityResult(int requestCode, int resultCode, Intent data)
        {
            super.onActivityResult(requestCode, resultCode, data);
        }
    
        private static native void notifyDeviceAttached(int fd);
        private static native void notifyDeviceDetached(int fd);
    
        private final BroadcastReceiver usbManagerBroadcastReceiver = new BroadcastReceiver()
        {
            public void onReceive(Context context, Intent intent)
            {
                try
                {
                    String action = intent.getAction();
    
                    Log.d(TAG, "INTENT ACTION: " + action);
    
                    if (ACTION_USB_PERMISSION.equals(action))
                    {
                        Log.d(TAG, "onUsbPermission");
    
                        synchronized (this)
                        {
                            UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
    
                            if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false))
                            {
                                if(device != null)
                                {
                                    int fd = connectToDevice(device);
                                    Log.d(TAG,"device file descriptor: " + fd);
                                    notifyDeviceAttached(fd);
                                }
                            }
                            else
                            {
                                Log.d(TAG, "permission denied for device " + device);
                            }
                        }
                    }
    
                    if (UsbManager.ACTION_USB_DEVICE_ATTACHED.equals(action))
                    {
                        Log.d(TAG, "onDeviceConnected");
    
                        synchronized(this)
                        {
                            UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
    
                            if (device != null)
                            {
                                manager.requestPermission(device, mPermissionIntent);
                            }
                        }
                    }
    
                    if (UsbManager.ACTION_USB_DEVICE_DETACHED.equals(action))
                    {
                        Log.d(TAG, "onDeviceDisconnected");
    
                        synchronized(this)
                        {
                            UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
    
                            int fd = connectedDevices.get(device.getDeviceId());
    
                            Log.d(TAG, "device: " + device.getDeviceId() + " disconnected. fd: " + fd);
    
                            notifyDeviceDetached(fd);
                            connectedDevices.remove(device.getDeviceId());
                        }
                    }
                }
                catch(Exception e)
                {
                    Log.d(TAG, "Exception: " + e);
                }
            }
        };
    
        private byte[] bytes = new byte[64];
        private static int i = 0;
    
        private int connectToDevice(UsbDevice device)
        {
            connection = manager.openDevice(device);
            // if we make this, kernel driver will be disconnected
            connection.claimInterface(device.getInterface(4), true);
    
            Log.d(TAG, "inserting device with id: " + device.getDeviceId() + " and file descriptor: " + connection.getFileDescriptor()) ;
            connectedDevices.put(device.getDeviceId(), connection.getFileDescriptor());
    
            return connection.getFileDescriptor();
        }
    
        private void checkForDevices()
        {
            HashMap<String, UsbDevice> deviceList = manager.getDeviceList();
            Iterator<UsbDevice> deviceIterator = deviceList.values().iterator();
    
            while(deviceIterator.hasNext())
            {
                UsbDevice device = deviceIterator.next();
    
                if (device.getVendorId()==vendorId && device.getProductId()==productId)
                {
                    Log.d(TAG, "Found a device: " + device);
    
                    manager.requestPermission(device, mPermissionIntent);
                }
            }
        }
    }
    
    

    couple important things to note about Java's side stuff:

    • these two rows will provide native bindings to C++ code:
        private static native void notifyDeviceAttached(int fd);
        private static native void notifyDeviceDetached(int fd);
    
    
    • this row have to grab proper interface of your USB device. In my case it is composite device, exposing audio and TTY. Surprisingly, inside USB device descriptor my TTY interface has number 0x3, but I had to claim interface #4, when asking Android... you've been warned
    connection.claimInterface(device.getInterface(4), true);
    
    1. create a C++ glue..... sorry my naming....
      glue.h:
    #ifndef GLUE_H
    #define GLUE_H
    
    #include <QObject>
    #include <QSerialPort>
    #include <QtAndroidExtras/QAndroidJniObject>
    #include <QtAndroidExtras/QAndroidJniEnvironment>
    #include <QtAndroid>
    #include <QCoreApplication>
    #include <QGuiApplication>
    #include <QRunnable>
    class Glue : public QObject, public QRunnable
    {
        Q_OBJECT
    public:
        explicit Glue() {}
        explicit Glue(int fd)  : fd(fd), running(true) {}
        void run();
    public slots:
        void stop();
    signals:
        void newSerialData(QByteArray ba);
    private:
        bool running;
        int fd;
    
    };
    
    extern Glue *glue;
    
    #endif // GLUE_H
    
    

    glue.c:

    #include "glue.h"
    #include <QSerialPortInfo>
    #include <QDebug>
    #include <QThreadPool>
    #include <sys/ioctl.h>
    #include <unistd.h>
    #include <errno.h>
    #include <linux/usbdevice_fs.h>
    
    Glue *glue;
    void Glue::stop()
    {
        running = false;
        setAutoDelete(true);
    }
    
    
    
    void Glue::run()
    {
    
        while(running)
        {
    
    #ifdef Q_OS_ANDROID
            struct usbdevfs_bulktransfer data;
            uint8_t buf[64] = {0};
            data.ep = 0x82;
            data.len = 64;
            data.data = buf;
            data.timeout = 1000;
            int res = ioctl(fd, USBDEVFS_BULK, &data);
    
            if(res > 0)
            {
                QByteArray ba = QByteArray((const char*)buf, res);
                emit newSerialData(ba);
            }
            qDebug() << "dfulog" <<  res << buf[0] << buf[1] << buf[2] << buf[3] << buf[4] << buf[5] << buf[6] << buf[7];
    #endif
        }
    }
    
    #include <QQmlApplicationEngine>
    #include <QQmlContext>
    
    extern QQmlApplicationEngine *engine;
    
    #ifdef __cplusplus
    extern "C" {
    #endif
    
    
    JNIEXPORT void JNICALL Java_com_mycompany_myapp_MyActivity_notifyDeviceAttached
    (JNIEnv *, jclass, jint fd)
    {
        glue = new Glue(fd);
    
        glue->moveToThread(engine->thread());
        engine->rootContext()->setContextProperty("glue", glue);
        glue->setAutoDelete(false);
        QThreadPool *threadPool = QThreadPool::globalInstance();
        threadPool->start(glue);
    }
    JNIEXPORT void JNICALL Java_com_mycompany_myapp_MyActivity_notifyDeviceDetached
    (JNIEnv *env, jclass obj, jint fd)
    {
        glue->setAutoDelete(true);
        glue->stop();
        QCoreApplication *qapp = QCoreApplication::instance();
        qapp->quit();
    }
    
    
    
    #ifdef __cplusplus
    }
    #endif
    
    

    Notes:

    • besides regular C++ stuff, we implement two native functions, those are to be called from Java's internals.... names are pretty strange, yeah... these names follow java's naming conventions for native code.

    • This is the second important thing to properly setup data receipt from your USB device:

    data.ep = 0x82;
    

    That is endpoint, which in my case is assigned to TX data from my device to the host (Android phone).

    • the core code, responsible to data acquisition from USB hardware is this ioctl (if OK, it returns number of bytes, grabbed from device):
    int res = ioctl(fd, USBDEVFS_BULK, &data);
    
    • QML engine and glue objects have to reside in the same thread, so I had to make engine global var, initialized in main.c
    extern QQmlApplicationEngine *engine;
    ....
        glue = new Glue(fd);
        glue->moveToThread(engine->thread());
    

    I'm not 100% sure it is required, but it somehow resides in my long-lasting project, so for the sake of completeness:

    add

    first.commands = \
         javah -d ../<YOUR PROJECT ROOT> -classpath ../<YOUR PROJECT ROOT>/android/src com.mycompany.myapp.MyActivity
    QMAKE_EXTRA_TARGETS += first
    
    

    to QT's .pro file. This will produce java's native code header com_mycompany_myapp_MyActivity.h:

    /* DO NOT EDIT THIS FILE - it is machine generated */
    #include <jni.h>
    /* Header for class com_eart_dfu_MyActivity */
    
    #ifndef _Included_com_eart_dfu_MyActivity
    #define _Included_com_eart_dfu_MyActivity
    #ifdef __cplusplus
    extern "C" {
    #endif
    /*
     * Class:     com_eart_dfu_MyActivity
     * Method:    notifyDeviceAttached
     * Signature: (I)V
     */
    JNIEXPORT void JNICALL Java_com_mycompany_myapp_MyActivity_notifyDeviceAttached
      (JNIEnv *, jclass, jint);
    
    /*
     * Class:     com_eart_dfu_MyActivity
     * Method:    notifyDeviceDetached
     * Signature: (I)V
     */
    JNIEXPORT void JNICALL Java_com_mycompany_myapp_MyActivity_notifyDeviceDetached
      (JNIEnv *, jclass, jint);
    
    #ifdef __cplusplus
    }
    #endif
    #endif
    
    

    for me, it just slightly helped to discover Android-NDK-QT relationships...

    1. update QT's auto-generated AndroidManifest.xml:
    • replace the activity name:
    <activity android:configChanges="orientation|uiMode|screenLayout|screenSize|smallestScreenSize|layoutDirection|locale|fontScale|keyboard|keyboardHidden|navigation" android:name="com.mycompany.myapp.MyActivity" android:label="-- %%INSERT_APP_NAME%% --" android:screenOrientation="unspecified" android:launchMode="singleInstance">
    
    • add device filter bindings:
                <intent-filter>
                    <action android:name="android.intent.action.MAIN"/>
                    <category android:name="android.intent.category.LAUNCHER"/>
                   <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"/>
                </intent-filter>
    
    
    • add USB OTG feature bindings:
    <uses-feature android:name="android.hardware.usb.host"/>
        <uses-sdk android:minSdkVersion="18" android:targetSdkVersion="16"/>
        <supports-screens android:largeScreens="true" android:normalScreens="true" android:anyDensity="true" android:smallScreens="true"/>
    
    1. create main.c:
    // javap -s -bootclasspath /opt/android-sdk/platforms/android-8/android.jar -classpath bin/classes android.app.Activity
    #include "mainwindow.h"
    #include <QApplication>
    #include "glue.h"
    
    #include <QGuiApplication>
    #include <QQmlApplicationEngine>
    #include <QFontDatabase>
    #include <QDebug>
    #include <QThreadPool>
    #include <QQmlContext>
    
    QQmlApplicationEngine *engine;
    
    int main(int argc, char *argv[])
    {
        QGuiApplication app(argc, argv);
        QGuiApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
        engine = new QQmlApplicationEngine();
        engine->load(QUrl("qrc:/MainWindow.qml"));
        if (engine->rootObjects().isEmpty())
            return -1;
        return app.exec();
    
    }
    
    

    Things to note: I couldn't manage it somehow else, so I've made engine a globe to be able to ref it from Java's native call. It is required to be able to promote Glue object&signal to QML's internals

    1. create MainWindow.qml:
    import QtQuick 2.8
    import QtQuick.Controls 2.2
    import QtQuick.Controls.Material 2.1
    
    ApplicationWindow {
    
        Component.onDestruction: {
            console.log("QML exit")
            Qt.quit()
        }
    
        id: window
        width: 300
        height: 400
        visible: true
        title: qsTr("Swipe to Remove")
     
        Column {
            anchors.fill: parent
    
    
            Rectangle {
                id: base
                width: parent.width
                height: parent.height / 3
                border.color: "black"
                border.width: 10
                radius: 15
            }
    
            Text {
                id: str
                text: qsTr("text")
                //        color: "red"
    
                font.pointSize: 16
    
    
            }
    
        }
        Connections {
            target: glue
            onNewSerialData: {
                str.text = ba
            }
        }
    }
    
    

    Notes: most important stuff is Connections, that listens Glue's signal, sending data, received from USB device. "ba" - is the name of signal's arg, used within its declaration in glue.h

    after hours of study, I had finally discovered that there is no other easy way to talk to USB device over than system's ioctl. In this case you don't have any matter of libusb or other driver's convenience, so you have to be familiar this basic USB internals: interface, endpoint, its type (bulk, isoc, interrupt, control)...


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.