基于强化学习的空调系统运行优化OPTIMIZATIONOF-同济大学

基于强化学习的空调系统运行优化

王尉同 1 潘毅群 1 黄治钟 2

1

(同济大学 机械与能源工程学院，上海 201804

2 同济大学 中德工程学院，上海 201804)

摘要：空调系统运行优化是建筑节能的重要组成部分。本文将强化学习算法应用在空调系

统运行优化中。首先建立空调系统运行优化任务的马尔科夫决策过程模型，接下来确立规

则对探索空间进行进一步缩减，使探索过程尽可能满足系统设备运行和被控区域舒适度的

约束，最后应用 Q-Learning 算法探索学习不断改进控制策略。通过仿真案例对算法进行了

测试。采用动态仿真软件 TRNSYS 和 MATLAB 进行联合仿真，对被控区域送风量进行了

优化，仿真结果显示，与基于固定规则的控制策略和 PID 控制相比，本文提出的基于强化

学习的控制方法可以在小幅提升被控区域舒适度的同时可将运行费用分别降低超过 7%和

4.5% 。

关键词：空调系统；运行优化；强化学习；Q-Learning

OPTIMIZATION OF HVAC SYSTEM

OPERATION BASED ON

REINFORCEMENT LEARNING

Abstract: The operation optimization of HVAC system is an essential part of building energy

conservation. In this work, reinforcement learning technique is applied to optimize HVAC system

operation. Firstly, we formulate the HVAC system operation as a Markov decision process.

Secondly, rules are defined to shrinkage the exploration space in order to avoid violating equipment

operation constraints and conditioned zones comfort constraints during exploration. Finally, Q-

Learning algorithm is deployed to improve the control strategy through interacting with the

environment. This study tests the algorithm by a simulation case. Dynamic simulation program

TRNSYS and MATLAB are utilized to conduct co-simulation. Results show that, compared with

fixed rule-based control strategy and PID controller, reinforcement learning controller can reduce

energy cost by over 7% and 4.5% separately by adjusting supply air flow rate, and at the same time

improve occupants comfort level slightly.

Keywords: HVAC system; operation optimization; reinforcement learning; Q-Learning

0. 引言

空调系统运行优化是建筑节能的重要组成部分。建筑运行优化的基本目标是在满足被

控区域舒适度要求的前提下尽可能的降低能耗或者能耗消费。为了适应室外不断变化的天

气情况和室内负荷变化情况，如何动态调整系统中的设定值或者运行规则从而提高建筑能

效一直是建筑运行优化的研究重点。

经典控制器包括基于规则的控制(如启停控制)，PID 控制等由于结构简单，初投资较

低被广泛应用，然而空调系统是典型的高度非线性、不确定性、时变性、耦合性的复杂系