The IoT testbed in UNSW includes various systems, roughly for 3 types of applications, which namely are wearable IoT devices, smart home and preliminary ambient intelligences.
The smart home is equipped with sensors in various locations that can be used to monitoring activities of daily living (ADLs) and in our proposed scenarios to help monitoring and proactively help individual requiring extra care.
The wearable sensors here include non-invasive electroencephalogram (EEG)-based Brain Computer Interfaces (BCI) and wearable Inertial Measurement Units (IMU). Such sensors can be applied in multiple scenarios such as to identify individuals, or to promptly guess(simple concepts such as shapes, letters, and basic body movements) what the person is thinking.
There are also some preliminary ambient intelligence devices, such as a set of small robots and small robotic arms.
Wearable IoT devices
Personnel identification
Biometric authentication involves various technologies to identify individuals by exploiting their unique, measurable physiological and behavioural characteristics. However, traditional biometric authentication systems (e.g., face recognition, iris, retina, voice, and fingerprint) are facing an increasing risk of being tricked by biometric tools such as anti-surveillance masks, contact lenses, vocoder, or fingerprint films. Here we tested to take the advantages of both EEG-and gait-based systems for fake resistance, we propose a multimodal biometric authentication system, to overcome the limitations of traditional unimodal biometric authentication systems. This authentication system contains three independent models: an Invalid ID Filter Model, a Delta band based EEG Identification Model, and a Gait Identification Model, to detect invalid EEG data, recognize the EEG ID and Gait ID, respectively.
Publication 1: https://dl.acm.org/doi/abs/10.1145/3393619
Publication 2: https://dl.acm.org/doi/10.1145/3264959
Code and dataset 1 : https://github.com/xiangzhang1015/DeepKey
Code and dataset 2: https://github.com/xiangzhang1015/MindID
Brain activity recognition
An EEG-based BCI enables people to communicate with the outside world by interpreting the EEG signals of their brains to interact with other IoT devices such as connected wheelchairs and intelligent robots. More specifically, motor imagery EEG (MI-EEG), which reflects a subject’s active intent, is attracting increasing attention for a variety of BCI applications. Accurate classification of MI-EEG signals while essential for effective operation of BCI systems is challenging due to the significant noise inherent in the signals and the lack of informative correlation between the signals and brain activities. Hence, we propose to use deep neural network to simultaneously learns robust high-level feature presentations through low-dimensional dense embeddings from raw MI-EEG signals.
An operational prototype of a brain typing system based on our proposed model, which demonstrates the efficacy and practicality of our approach. A video demonstrating the system is made available.
Publication: https://ieeexplore.ieee.org/abstract/document/8444575
Code and dataset: https://github.com/xiangzhang1015/Brain_typing
Video: https://www.youtube.com/watch?v=Dc0StUPq61k
We also investigated to reconstruct the geometrical shape based on the brain signals, and proposed to learning the latent discriminative representation of the raw EEG signals, and then, based on the learned representation, we propose an adversarial reconstruction framework to recover the geometric shapes which are visualizing by the human.
Publication : http://ajiips.com.au/papers/V15.2/v15n2_40-47.pdf
Code and dataset : https://github.com/xiangzhang1015/EEG_Shape_Reconstruction
Ambient Intelligence
Internet of Things (IoT) enables the connection of a broad range of artifacts with advanced sensory technologies and produces massive amounts of data to support ambient intelligence. xWhile the potential of IoT systems is widely recognized, there is still limited work to demonstrate such a system with the autonomy and the ability to execute in the real world. Inspired by the successful introduction of robots to specialized IoT environments, we propose an end-to end solution for a generic, interactive ambient intelligence system, where robotic assistants can assist humans in conducting activities in IoT-enabled smart homes.
We have developed 2 preliminary demos based on our study.
Firstly we demonstrates the possibility of robot navigates to the required unknown location as well as exploration and survey of the unknown area for map building. The robot equips 360° laser distance sensor for SLAM and navigation.
Video 1 : https://youtu.be/Aii3XGDVv-Y
Then we shows a scenario of domestic robot as intelligent assistant which intimately helps the subject with tasks. The system was trained to help certain tasks, such as pass spoon when bowl appears as the subject preparing food. Optical flow was used to predict the trajectory of active moving objects with RGB-D camera in 3D space.
Video 2 : https://drive.google.com/file/d/1zpC60PhC_v7l-p2YbraDSLANX5gf15D3/view
Publication 1: https://ieeexplore.ieee.org/abstract/document/9889122/