Please wait a minute...
img

官方微信

遥感技术与应用  2020, Vol. 35 Issue (2): 478-483    DOI: 10.11873/j.issn.1004-0323.2020.2.0478
遥感应用     
基于Python的生态监测物联网数据自动采汇中间件应用研究
胡飞虎1,2(),郭建文1,3(),吴阿丹1,2,杨鹏飞1,李亚珍1,2
1.中国科学院西北生态环境资源研究院,甘肃 兰州 730000
2.中国科学院大学,北京 100049
3.江苏省地理信息资源开发与利用协同创新中心,江苏 南京 210023
Application Research of Cata Automatic Collection and Aggregation Middleware for Ecological Monitoring IoT based on Python
Feihu Hu1,2(),Jianwen Guo1,3(),Adan Wu1,2,Pengfei Yang1,Yazhen Li1,2
1.Northwest Institute of Eco-Environment and Resources, Chinese Academy of Sciences, Lanzhou 730000, China
2.University of Chinese Academy of Sciences, Beijing 100049, China
3.Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing 210023, China
 全文: PDF(1354 KB)   HTML
摘要:

针对生态监测物联网数据自动获取和采集过程中的应用需求,参考此前研发的相关子系统,分析了其原生的不足之处,用面向对象理念进行了全新设计,使其以多源异构数据自动采汇中间件的形式呈现,被后端的观测数据自动综汇平台调用。该中间件具有灵活的可扩展性,能够与不同厂商、不同来源的物联网数据采集设备协作,获取实时的多源异构监测数据,然后对数据进行归一化处理后,自动汇总存入监测数据库。该中间件运用模块化的软件工程理念设计,主要由数据自动获取模块、数据自动解析处理模块、数据自动入库模块这3部分组成,模块之间高内聚、低耦合,以数据流为纽带,紧密配合,完成整个监测数据流采集入库的全自动化处理流程。与原有子系统相比较,其具有更明确的模块划分、更高的灵活性和更好的可维护性。该中间件采用简洁高效的Python语言实现,完全采用面向对象编程思想,所有类的设计遵循单一职责原则,面向接口编程,保证了程序具备稳定的功能和灵活的扩展性。该中间件经过充分测试及几个月的试运行,能够满足现有生态监测物联网监测数据的自动获取、解析处理和入库的业务需求。对于野外地理环境和生态环境的监测,只要涉及类似的数据采集处理流程,本文设计和实现的数据自动采汇中间件都具有一定的参考意义和应用价值。

关键词: 生态监测物联网(IOT)中间件Python自动入库数据自动采汇    
Abstract:

According to the application requirements in the process of data automatic acquisition and collection of the ecological monitoring Internet of Things. This paper refers to the related systems and developed a multi-source heterogeneous data automatic collection and aggregation middleware. The middleware had flexible scalability, it could cooperate with data sensing and collecting devices of ecological monitoring Internet of things from different manufacturers, obtain multi-source heterogeneous real-time data, then automatically stored the data into database after normalization. Based on modular design concept, the middleware was composed of three modules: data automatic collecting module, data automatic parsing processing module and data automatic storage module. These modules had high cohesion and low coupling, and closely cooperated to complete the full automatic processing of the monitoring data flow. The middleware was implemented with Python, which were fully used the object-oriented programming. The design of the class was followed by a single responsibility principle and interfaces-oriented programming, which ensured the top-down inheritance and extensibility of the program. The middleware has been fully tested for several months, and it could accomplish the business requirements of monitoring data collecting, parsing processing and automatic warehousing of the ecological monitoring Internet of Things. For the ecological monitoring IoT system, the data automatic collection and aggregation middleware has considerable reference significance and application value.

Key words: Ecological monitoring    Internet of Things    Middleware    Python    Automatic warehousing    Data automatic collection and aggregation
收稿日期: 2018-12-05 出版日期: 2020-07-10
ZTFLH:  TP75  
基金资助: 国家重点研发计划项目(2016YFC0500105)
通讯作者: 郭建文     E-mail: hufeihu@lzb.ac.cn;guojw@lzb.ac.cn
作者简介: 胡飞虎(1992-),男,甘肃陇南人,硕士研究生,主要从事生态监测物联网应用研究。E?mail:hufeihu@lzb.ac.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
胡飞虎
郭建文
吴阿丹
杨鹏飞
李亚珍

引用本文:

胡飞虎,郭建文,吴阿丹,杨鹏飞,李亚珍. 基于Python的生态监测物联网数据自动采汇中间件应用研究[J]. 遥感技术与应用, 2020, 35(2): 478-483.

Feihu Hu,Jianwen Guo,Adan Wu,Pengfei Yang,Yazhen Li. Application Research of Cata Automatic Collection and Aggregation Middleware for Ecological Monitoring IoT based on Python. Remote Sensing Technology and Application, 2020, 35(2): 478-483.

链接本文:

http://www.rsta.ac.cn/CN/10.11873/j.issn.1004-0323.2020.2.0478        http://www.rsta.ac.cn/CN/Y2020/V35/I2/478

图1  生态监测物联网的基本架构
图2  数据自动获取模块原理图
图3  数据自动解析处理模块原理图
图4  数据自动入库模块原理图
1 Chen Rui. IOT:A Strategic Opportunity for National Innovative Development in Post IP Era[J]. Bulletin of Chinese Academy of Sciences, 2010, 25(1): 41-49.
1 陈锐. 物联网——后IP时代国家创新发展的重大战略机遇[J]. 中国科学院院刊, 2010, 25(1): 41-49.
2 Wu Danna, Jing Hong, Zhang Jinmeng, et al. Application of Internet of Things in Environmental Monitoring[J]. Journal of Anhui Agricultural Sciences, 2014, 42(10): 3076-3079.
2 吴丹娜,江洪,张金梦,等. 环境监测中物联网技术的应用[J]. 安徽农业科学, 2014, 42(10): 3076-3079.
3 Luo Guilan, Deng Shourong, Zhang Mei, et al. Research On Ecological Environment Monitoring Methods of Erhai Lake based on Internet of Things[J]. Journal of Dali University, 2013, 12(4): 23-28.
3 罗桂兰,邓寿容,张梅,等. 基于物联网的洱海生态环境监测方法研究[J]. 大理学院学报, 2013, 12(4): 23-28.
4 Wang Xijie. Application Research of Ecological Environment Monitoring based on Internet of Things Technology[J]. Transducer and Microsystem Technologies, 2011, 30(7): 149-152.
4 王希杰. 基于物联网技术的生态环境监测应用研究[J]. 传感器与微系统, 2011, 30(7): 149-152.
5 Li Xiaoxue. The Internet of Things Technology in the Application of Ecological Environmental Monitoring Analysis[J]. New Technology & New Process, 2015, 24(8): 72-75.
5 李晓雪. 物联网技术在生态环境监测中的应用分析[J]. 新技术新工艺, 2015, 24(8): 72-75.
6 Li Xin, Liu Shaomin, Sun Xiaomin, et al. Innovative Development of Equipments and Internet-of-things Techniques for Ecosystem Monitoring and Its Demonstration[J]. Acta Ecologica Sinica, 2016, 36(22): 7023-7027.
6 李新,刘绍民,孙晓敏,等. 生态系统关键参量监测设备研制与生态物联网示范[J]. 生态学报, 2016, 36(22): 7023-7027.
7 Li Xin, Liu Shaomin, Ma Mingguo, et al. HiWATER: An Inte-grated Remote Sensing Experiment on Hydrological and Ecological Processes in the Heihe River Basin[J]. Advances in Earth Science, 2012, 27(5): 481-498.
7 李新,刘绍民,马明国,等. 黑河流域生态—水文过程综合遥感观测联合试验总体设计[J]. 地球科学进展, 2012, 27(5): 481-498.
8 Guo Jianwen, Chang Hailong, Shang Qingsheng. Design and Implement of Normalization Data Interface and Automatic Data Warehousing System for Heterologous WSN Instruments[J]. Remote Sensing Technology and Application, 2013, 28(3): 405-410.
8 郭建文,常海龙,尚庆生. 异源WSN观测仪器归一化数据接口及自动入库系统的设计与实现[J]. 遥感技术与应用, 2013, 28(3): 405-410.
9 Guo Jianwen, Shang Qingsheng, Chang Hailong. Design of Field Observation Data Automatic Assembling System[J]. Remote Sensing Technology and Application, 2013, 28(3): 399-404.
9 郭建文,尚庆生,常海龙,等. 野外观测数据自动综汇系统方案设计[J].遥感技术与应用,2013,28(3):399-404.
10 Wu Adan, Guo Jianwen, Li Jianxuan, et al. The Application of Web-based Visualization System for a Wireless Sensor Network in Heihe Watershed Allied Telemetry [J]. Remote Sensing Technology and Application, 2013, 28(3): 416-422.
10 吴阿丹,郭建文,李建轩,等. 基于Web的黑河流域生态水文WSN自动观测数据可视化系统应用研究[J]. 遥感技术与应用, 2013, 28(3): 416-422.
11 Cheng Chunrui, Liu Wanjun. Construction of High Coheres and Low Coupling Software Architecture[J]. Computer Systemsand Applications, 2009(7): 19-22.
11 程春蕊,刘万军. 高内聚低耦合软件架构的构建[J]. 计算机系统应用, 2009(7): 19-22.
[1] 吴阿丹,郭建文. 遥感影像Web共享关键技术应用研究[J]. 遥感技术与应用, 2016, 31(6): 1209-1214.
[2] 郭建文,常海龙,尚庆生. 异源WSN观测仪器归一化数据接口及自动入库系统的设计与实现[J]. 遥感技术与应用, 2013, 28(3): 405-410.
[3] 汪超亮,唐伶俐. 遥感卫星数据编目存档系统兼容性探讨[J]. 遥感技术与应用, 2007, 22(3): 428-432.
[4] 王春晓,李佳田,王定勇,刘洪斌,武 伟. 集成型空间数据库技术探讨与应用实例[J]. 遥感技术与应用, 2003, 18(2): 109-114.
[5] 范建容 周万村 高世忠 郭洁. 遥感与模糊评判在森林火灾后生态监测评价中的应用[J]. 遥感技术与应用, 1995, 10(4): 42-47.