Application and performance optimization of python libraries in real-time data processing for IoT electronic devicesabstract
Yanrui Chen
Shanghai Lida University
DOI: https://doi.org/10.59429/esta.v12i2.10573
Keywords: IoT; Python tool set; Telemetry dataset
Abstract
Owing to the prevalent growth of internet of things (IoT) electronics devices, there has been an increasing requirement for processing real - time data and optimizing it in a faster manner. In this paper, a low - overhead and a pragmatic python - based tool set is proposed that can perform processing and optimizing of streaming data generated by IoT sensors. By integrating mature open-source libraries (e.g., pandas, NumPy, and scikit-learn), we constructed a real-time feature engineering framework that enables asynchronous data cleaning, feature extraction, and dynamic tuning of model parameters. System validation was conducted using the public UCI IoT telemetry dataset, with quantitative analyses of core performance metrics including system latency, memory requirements, and signal throughput. Furthermore, the accompanying Python code examples demonstrate specific implementations of data parsing, anomaly detection, and adaptive signal filtering.
References
[1]UCI Machine Learning Repository: IoT Telemetry Dataset
[2]Wes McKinney. (2010). Data Structures for Statistical Computing in Python.
[3]Pedregosa et al. (2011). Scikit-learn: Machine Learning in Python.
[4]Van Rossum, G., & Drake, F. L. (2009). Python 3 Reference Manual.
[5]Kim, J., & Hong, S. (2019). Lightweight anomaly detection in sensor streams for IoT.
[6]Chen Yanrui. Real-Time Dynamic Report Construction and Multidimensional Data Analysis System V1.0. National Copyright Administration of China, 2025. Registration No. 2025SR0292356.