Fall’26: CS 6301 Special Topics in CS: Machine Learning in Mobile Computing

Course Info

Instructor: Yi Ding

Office: ECSS 4.703

Office hours: by appointment

Email: yi.ding@utdallas.edu

Lecture: 8:30 pm - 9:45 pm, Monday/Wednesday

Location: JSOM 2.714

Syllabus

Project.

Assignment.

Course Description

Machine learning is transforming the way mobile and embedded systems perceive and interact with the world. Empowered by rich data from sensors embedded in our phones, wearables, vehicles, and infrastructure, mobile computing is becoming increasingly intelligent, context-aware, and human-centric.

In this course, we explore how sensing technologies, machine learning techniques, and mobile systems jointly enable applications in wireless sensing, multimodal fusion, on-device learning, and adaptive edge intelligence. Topics include RF-based sensing (e.g., Wi-Fi, Bluetooth, GPS, satellite), acoustic and visual sensing, inertial and environmental sensing, signal tokenization and feature modeling, mobile system optimization, privacy-preserving learning, and the use of foundation models in mobile and sensing scenarios.

Students are expected to:

(i) read and present research papers from top-tier conferences (e.g., MobiCom, SenSys, UbiComp, NeurIPS),
(ii) participate actively in in-class discussions and invited talks from academia and industry, and
(iii) design, implement, and present a final project that explores new ideas in mobile sensing and machine learning.

Course Learning Objectives

By the end of this course, you will be able to

1. Understand the core principles of applying machine learning techniques to mobile and embedded systems, including sensing modalities, signal processing, and on-device learning.
2. Explain state-of-the-art research and system designs in mobile sensing, multimodal data fusion, edge intelligence, and federated learning.
3. Evaluate the trade-offs and constraints in mobile and resource-constrained environments (e.g., latency, energy, privacy), and how they affect the deployment of machine learning models.
4. Design and propose intelligent mobile sensing systems that integrate machine learning, signal modeling, and system-level optimization.
5. Implement prototypes or simulations using real-world or simulated sensor data, employing tools such as Python, PyTorch/TensorFlow, and edge deployment toolkits.
6. Communicate technical insights effectively through paper presentations, invited talk discussions, and final project demos.

Required Texts

No books are required. All the materials will be online.

Course Schedule (Tentative)

W1 Course Introduction & Motivation (08/24, 08/26, 08/31)

• Lecture: Course Introduction
• Lecture: Paper Reading and Presentation
• Lecture: ML in Mobile Computing - Background

W2 Sensing Modalities I: GPS and Satellite (09/02, 09/09)

• Lecture: GPS and Satellite

• Guest Talk

• Paper 1 Dong, Huixin, et al. “Gpsmirror: Expanding accurate gps positioning to shadowed and indoor regions with backscatter.” Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.
• Paper 2 Dong, Huixin, et al. “Gpsense: Passive sensing with pervasive gps signals.” Proceedings of the 30th Annual International Conference on Mobile Computing And Networking. 2024.
• Paper 3 Rathi, Raghav, and Zhenghao Zhang. “StarAngle: User Orientation Sensing with Beacon Phase Measurements of Multiple Starlink Satellites.” Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems. 2024.
• Paper 4 Ecola, Geneva, et al. “SARLink: Satellite Backscatter Connectivity using Synthetic Aperture Radar.” Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems. 2025.
• Paper 5 Hong, Zhiqing, et al. “Smallmap: Low-cost community road map sensing with uncertain delivery behavior.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8.2 (2024): 1-26.

W3 Sensing Modalities II: Wi-Fi and Bluetooth (09/14, 09/16)

• Lecture: Wi-Fi and Bluetooth

• Paper 1 Ding, Jian, et al. “Cost-effective soil carbon sensing with wi-fi and optical signals.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper 2 Zheng, Yue, et al. “Zero-effort cross-domain gesture recognition with Wi-Fi.” Proceedings of the 17th annual international conference on mobile systems, applications, and services. 2019.
• Paper 3 Ni, Jiazhi, et al. “Experience: Pushing indoor localization from laboratory to the wild.” Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. 2022.
• Paper 4 Li, Xin, et al. “Uwb-fi: Pushing wi-fi towards ultra-wideband for fine-granularity sensing.” Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services. 2024.
• Paper 5 Adib, Fadel, and Dina Katabi. “See through walls with WiFi!.” Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM. 2013.
• Paper 6 Wang, Yuxi, Kaishun Wu, and Lionel M. Ni. “Wifall: Device-free fall detection by wireless networks.” IEEE Transactions on Mobile Computing 16.2 (2016): 581-594.

W5 Sensing Modalities III: IMU and Environmental Sensors (09/21, 09/23)

• Paper 1 Xu, Huatao, et al. “Limu-bert: Unleashing the potential of unlabeled data for imu sensing applications.” Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 2021.
• Paper 2 Xu, Huatao, et al. “Practically adopting human activity recognition.” Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.
• Paper 3 Zhang, Xiyuan, et al. “Unimts: Unified pre-training for motion time series.” Advances in Neural Information Processing Systems 37 (2024): 107469-107493.
• Paper 4 Zhou, Pengfei, et al. “Iodetector: A generic service for indoor outdoor detection.” Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems. 2012.
• Paper 5 Brooks, Jas, and Pedro Lopes. “Smell & paste: Low-fidelity prototyping for olfactory experiences.” Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023.

W4 Sensing Modalities IV: Acoustic and Visual (09/28, 09/30)

• Paper 1 Wang, Wei, Alex X. Liu, and Ke Sun. “Device-free gesture tracking using acoustic signals.” Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. 2016.
• Paper 2 Cao, Shirui, et al. “Powerphone: Unleashing the acoustic sensing capability of smartphones.” Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.
• Paper 3 Zhang, Yongzhao, et al. “Acoustic sensing and communication using metasurface.” 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). 2023.
• Paper 4 Zhang, Yanbo, et al. “Face recognition in harsh conditions: An acoustic based approach.” Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services. 2024.
• Paper 5 Li, Ke, et al. “Gazetrak: Exploring acoustic-based eye tracking on a glass frame.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper 6 Li, Yuanqi, et al. “Reducto: On-camera filtering for resource-efficient real-time video analytics.” Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication. 2020.

W6 Sensing Modalities V: UWB and mmWave (10/05, 10/07)

• Paper 1 Großwindhager, Bernhard, et al. “Snaploc: An ultra-fast uwb-based indoor localization system for an unlimited number of tags.” Proceedings of the 18th International Conference on Information Processing in Sensor Networks. 2019.
• Paper 2 Wang, Ziqi, et al. “Uwhear: Through-wall extraction and separation of audio vibrations using wireless signals.” Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 2020.
• Paper 3 Zhou, Hao, et al. “Rethinking orientation estimation with smartphone-equipped ultra-wideband chips.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper 4 Lien, Jaime, et al. “Soli: Ubiquitous gesture sensing with millimeter wave radar.” ACM Transactions on Graphics (TOG) 35.4 (2016): 1-19.
• Paper 5 Xue, Hongfei, et al. “Mmmesh: Towards 3d real-time dynamic human mesh construction using millimeter-wave.” Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services. 2021.
• Paper 6 Pallaprolu, Anurag, et al. “Crowd analytics with a single mmwave radar.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.

W7 Sensing Modalities VI: Multi-Modality (10/12, 10/14)

• Paper1 Yao, Shuochao, et al. “Deepsense: A unified deep learning framework for time-series mobile sensing data processing.” Proceedings of the 26th International Conference on World Wide Web. 2017.
• Paper2 Ouyang, Xiaomin, et al. “Cosmo: Contrastive fusion learning with small data for multimodal human activity recognition.” Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. 2022.
• Paper3 Ouyang, Xiaomin, et al. “Admarker: A multi-modal federated learning system for monitoring digital biomarkers of Alzheimer’s disease.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper4 Liu, Yimeng, et al. “Hydra: Accurate multi-modal leaf wetness sensing with mm-wave and camera fusion.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper5 Dai, Shenghong, et al. “Babel: A scalable pre-trained model for multi-modal sensing via expandable modality alignment.” Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems. 2025.
• Paper6 Post, Kevin, et al. “Contextllm: Meaningful context reasoning from multi-sensor and multi-device data using llms.” Proceedings of the 26th International Workshop on Mobile Computing Systems and Applications. 2025.

W8 Project Proposal Presentation (10/19, 10,21)

W9 Foundation Models for Sensing (10/26, 10/28)

• Paper1 Xu, Huatao, et al. “Penetrative ai: Making llms comprehend the physical world.” Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications. 2024.
• Paper2 Imran, Sheikh Asif, et al. “Llasa: Large multimodal agent for human activity analysis through wearable sensors.” arXiv preprint arXiv:2406.14498 (2024).
• Paper3 Narayanswamy, Girish, et al. “Scaling wearable foundation models.” The Thirteenth International Conference on Learning Representations. 2025.
• Paper4 Quan, Pengrui, et al. “Sensorbench: Benchmarking llms in coding-based sensor processing.” Proceedings of the 26th International Workshop on Mobile Computing Systems and Applications. 2025.
• Paper5 Dong, Qifei, Xiangliang Chen, and Mahadev Satyanarayanan. “Creating edge ai from cloud-based llms.” Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications. 2024.
• Paper6 Yang, Bufang, et al. “Edgefm: Leveraging foundation model for open-set learning on the edge.” Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems. 2023.
• Paper7 Liu, Kaiwei, et al. “Tasksense: A translation-like approach for tasking heterogeneous sensor systems with llms.” Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems. 2025.

W10 Efficient Mobile AI (11/02, 11/04)

• Paper1 Han, Song, Huizi Mao, and William J. Dally. “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.” The Fourth International Conference on Learning Representations. 2016.
• Paper2 Frantar, Elias, et al. “Gptq: Accurate post-training quantization for generative pre-trained transformers.” The Eleventh International Conference on Learning Representations. 2023.
• Paper3 Lin, Ji, et al. “Awq: Activation-aware weight quantization for on-device llm compression and acceleration.” Proceedings of Machine Learning and Systems 6 (2024): 87-100.
• Paper4 Gu, Yuxian, et al. “Minillm: Knowledge distillation of large language models.” The Twelfth International Conference on Learning Representations. 2024.
• Paper5 Rastikerdar, Mohammad Mehdi, et al. “Cactus: Dynamically switchable context-aware micro-classifiers for efficient iot inference.” Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services. 2024.
• Paper6 Hojjat, Ali, et al. “Limitnet: Progressive, content-aware image offloading for extremely weak devices & networks.” Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services. 2024.
• Paper7 Lin, Ji, et al. “On-device training under 256kb memory.” Advances in Neural Information Processing Systems 35 (2022).

W11 Privacy, Security & Trust in Mobile Sensing (11/09, 11/11)

• Paper1 Zhang, Guoming, et al. “Dolphinattack: Inaudible voice commands.” Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017.
• Paper2 Ba, Zhongjie, et al. “Learning-based practical smartphone eavesdropping with built-in accelerometer.” Network and Distributed System Security Symposium (NDSS). 2020.
• Paper3 Shi, Cong, et al. “Face-mic: Inferring live speech and speaker identity via subtle facial dynamics captured by AR/VR motion sensors.” Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 2021.
• Paper4 Basak, Suryoday, and Mahanth Gowda. “Mmspy: Spying phone calls using mmwave radars.” 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.
• Paper5 McMahan, Brendan, et al. “Communication-efficient learning of deep networks from decentralized data.” Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. PMLR, 2017.
• Paper6 Abadi, Martin, et al. “Deep learning with differential privacy.” Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. 2016.
• Paper7 Bonawitz, Keith, et al. “Practical secure aggregation for privacy-preserving machine learning.” Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017.

W12 System-Level Co-Design & Optimization (11/16, 11/18)

• Paper1 Kang, Yiping, et al. “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge.” Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems. 2017.
• Paper2 Lane, Nicholas D., et al. “Deepx: A software accelerator for low-power deep learning inference on mobile devices.” 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). IEEE, 2016.
• Paper3 Fang, Biyi, Xiao Zeng, and Mi Zhang. “Nestdnn: Resource-aware multi-tenant on-device deep learning for continuous mobile vision.” Proceedings of the 24th Annual International Conference on Mobile Computing and Networking. 2018.
• Paper4 Lin, Ji, et al. “Mcunet: Tiny deep learning on iot devices.” Advances in Neural Information Processing Systems 33 (2020): 11711-11722.
• Paper5 Yuan, Jinliang, et al. “Mobile foundation model as firmware.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper6 Liu, Renyuan, et al. “DAF: An efficient end-to-end dynamic activation framework for on-device DNN training.” Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services. 2025.
• Paper7 Misra, Ashitabh, Nurani Saoda, and Tarek Abdelzaher. “Latency-constrained input-aware quantization of time series inference workflows at the edge.” IEEE INFOCOM 2025-IEEE Conference on Computer Communications. IEEE, 2025.

W13 Mobile AI Applications & Real-World Deployment (11/23, 11/25)

• Paper1 Wen, Hao, et al. “Autodroid: Llm-powered task automation in android.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.
• Paper2 Chen, Kaixin, et al. “Lit: Fine-grained toothbrushing monitoring with commercial led toothbrush.” Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.
• Paper3 Jiao, Wenli, et al. “Bioscatter: Low-power sweat sensing with backscatter.” Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services. 2023.
• Paper4 Zhang, Xi, et al. “mmfer: Millimetre-wave radar based facial expression recognition for multimedia iot applications.” Proceedings of the 29th Annual International Conference on Mobile Computing and Networking. 2023.
• Paper5 Wang, Zheng, et al. “Hearfire: Indoor fire detection via inaudible acoustic sensing.” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6.4 (2022): 1-25.
• Paper6 Cai, Chao, et al. “Acute: Acoustic thermometer empowered by a single smartphone.” Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 2020.
• Paper7 Sun, Zehua, et al. “Rf-egg: An rf solution for fine-grained multi-target and multi-task egg incubation sensing.” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 2024.

W14 Fall Break (11/30, 12/2)

W15 Final Project Presentation (12/7, 12/9)

Invited Talks

• Sep. 9, 2026, Anlan Yu (Peking University)

• Sep. 21, 2026, Zhiqing Hong (HKUST(GZ)

• Oct. 7, 2026, Fangwei Zhang (MSU)