US2026071771A1PendingUtilityA1
Optimized control method and system of building air conditionings taking personalized comfort of regional users into consideration
Est. expirySep 11, 2044(~18.1 yrs left)· nominal 20-yr term from priority
Inventors:WANG XINLILIU YINUOYIN XIAOHONGZHANG CHENGHAOXIAO YONGLIN CHENGENG FANGWANG LEIJIA LEIXU YONG
F24F 11/62F24F 11/46F24F 2120/20F24F 11/64Y02B30/70F24F 11/47F24F 11/63
71
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
An optimized control method of building air conditionings taking personalized comfort of regional users into consideration, comprising: describing a heating, ventilation and air conditioning system control problem as a Markov decision process, and defining a state, an action and a reward of each agent; learning by adopting a meta-learning method to obtain a user comfort reward function initial model; performing differentiated training on the user comfort reward function initial model by respectively using comfort feedback data from different regional users to obtain a differentiated comfort reward function model of each agent; and obtaining a reward function of each agent based on an energy consumption reward model and the differentiated comfort reward function model of each agent, training an actor network and a critic network of each agent, and making control decisions for a building heating, ventilation and air conditioning system by using the trained actor network.
Claims
exact text as granted — not AI-modified1 . An optimized control method of building air conditionings taking personalized comfort of regional users into consideration, comprising:
acquiring comfort feedback data from different regional users of a building and historical operation data of a building air conditioning system, and performing data preprocessing; describing a heating, ventilation and air conditioning system control problem as a Markov decision process (MDP), and defining each agent, and a state, an action and a reward of each agent; performing initial parameter leaning of a comfort reward function for a plurality of regions based on the comfort feedback data from a plurality of regional users by using a meta-learning method to obtain a user comfort reward function initial model; performing differentiated training and parameter updating on the user comfort reward function initial model by respectively using the comfort feedback data from the different regional users to obtain a differentiated comfort reward function model of each agent; and obtaining a reward function of each agent based on an energy consumption reward model and the differentiated comfort reward function model of each agent, training an actor network and a critic network of each agent, and making control decisions for the building heating, ventilation and air conditioning system by using the trained actor network; wherein: using a meta-learning method to obtain a user comfort reward function initial model specifically comprises: defining the comfort reward function of each regional user as a subtask; acquiring a plurality of sets of indoor environment states and comfort feedback data for the regional user that each subtask targets, and dividing into a Support set and a Query set; respectively training each subtask by using data in the Support set, and updating parameters of each subtask through gradient descent; performing testing in the Query set by using the parameters obtained through training, and then calculating a loss function under each subtask; averaging the loss functions of a plurality of subtasks, and updating an initial parameter of a meta-learning training task through gradient descent; calculating the loss function:
L
θ
,
b
=
1
K
∑
k
=
1
K
(
f
(
S
in
,
k
)
-
f
^
θ
(
S
in
,
k
)
)
2
;
wherein, ƒ(S in,k ) represents actual comfort feedback data from indoor users regarding indoor environment states obtained through sampling; {circumflex over (ƒ)} θ (S in,k ) represents a predicted output of a model represented by a weight parameter θ for the same input data S in,k ; K represents that the Support set totally contains K sets of data, and k represents a k(1≤k≤K) th set of data; S in,k represents a k th set of indoor environment data in the Support set; b represents a b th training subtask, 1≤b≤B;
updating parameters through gradient descent:
θ
′
=
θ
-
α
∇
θ
L
θ
,
b
;
wherein, θ′ is an updated weight parameter of a meta-learning network;
calculating the loss function under each subtask:
L
θ
′
,
b
=
1
K
′
∑
k
′
=
1
K
′
(
f
(
S
in
,
k
′
)
-
f
^
θ
″
(
S
in
,
k
′
)
)
2
;
wherein, S in,k′ is a k′ th set of indoor environment data in the Query set; θ″ represents a model weight parameter obtained during testing by using the Query set after training with the Support set; K′ represents that the Query set totally contains K′ sets of data; k′ represents a k′ th (1≤k′≤K′) set of data; and
averaging the loss functions of all users, and updating an initial parameter through gradient descent:
θ
=
θ
-
β
·
1
B
∑
b
=
1
B
∇
θ
L
θ
′
,
b
;
wherein, b represents a b th training subtask, 1≤b≤B.
2 . The optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 1 , wherein the comfort feedback data refer to different feeling feedbacks of the users regarding an indoor environment, the indoor environment comprises air temperature, relative humidity, and CO 2 concentration; the historical operation data of the building air conditioning system comprises building environment data and HVAC system operation data, and the building environment data comprises indoor temperature, indoor humidity, indoor CO 2 concentration, outdoor temperature, outdoor humidity, wind speed, and solar radiation; the HVAC system operation data comprise fan speed, chilled water temperature, and heating water temperature.
3 . The optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 1 , wherein considering a building with N regions, the N regions are defined as N regional agents, and an air handling unit AHU for the entire building is defined as an (N+1) th agent AHU;
a state comprising indoor temperature, indoor humidity and indoor CO 2 concentration is defined; an action comprising regional mass flow rate, fresh air volume and chilled water flow rate is defined; and a reward comprising energy consumption and comfort is defined.
4 . The optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 3 , wherein the defined state of each agent comprises:
for N regional agents, a state s i,t of each region i at time t is:
s
i
,
t
=
(
T
t
out
,
T
i
,
t
,
T
-
i
,
t
,
λ
t
,
k
i
,
t
,
C
i
,
t
,
H
i
,
t
)
;
wherein,
1
≤
i
≤
N
;
T
t
out
represents outdoor temperature, T i,t represents indoor temperature of the region i, T −i,t represents indoor temperature of an adjacent region of the region i, λ t represents electricity price, k i,t represents the number of residents in the region i, C i,t represents indoor CO 2 concentration of the region i, and Hit represents indoor humidity of the region i;
a state s N+1,t of the (N+1) th agent AHU at time t is expressed as:
s
N
+
1
,
t
=
(
λ
t
,
k
1
,
t
,
…
,
k
N
,
t
,
C
1
,
t
,
…
,
C
N
,
t
,
H
1
,
t
,
…
,
H
N
,
t
)
;
an action a i,t of each regional agent i(1≤i≤N) at time t is defined:
a
i
,
t
=
(
M
i
,
t
)
;
wherein, M i,t represents regional mass flow rate of the agent i(1≤i≤N);
an action a N+1,t of the (N+1) th agent AHU at time t is defined:
a
N
+
1
,
t
=
(
d
t
,
W
t
)
;
wherein d t represents fresh air volume, and W t represents chilled water flow rate;
for N regional agents, a reward function r i,t is:
r
i
,
t
=
λ
1
r
i
,
t
com
(
s
i
,
t
,
a
i
,
t
)
+
λ
2
r
i
,
t
e
n
(
s
i
,
t
,
a
i
,
t
)
;
wherein,
r
i
,
t
c
o
m
(
s
i
,
t
,
a
i
,
t
)
represents a comfort penalty of a regional agent i,
r
i
,
t
e
n
(
s
i
,
t
,
a
i
,
t
)
represents an energy consumption penalty of the regional agent i, and λ 1 and λ 2 are positive weight coefficients; s i,t is a state of the regional agent i at time t, and a i,t is an action of the regional agent i at time t;
for the (N+1) th agent AHU, a reward function r N+1,t is:
r
N
+
1
,
t
=
λ
3
∑
i
=
1
N
r
i
,
t
com
(
s
i
,
t
,
a
i
,
t
)
+
λ
4
r
N
+
1
,
t
en
(
s
N
+
1
,
t
,
a
N
+
1
,
t
)
;
wherein
r
N
+
1
,
t
e
n
(
S
N
+
1
,
t
,
a
N
+
1
,
t
)
represents an energy consumption penalty of the (N+1) th agent AHU; λ 3 and λ 4 are positive weight coefficients; s N+1,t is a state of the (N+1) th agent AHU at time t; a N+1,t is an action of the (N+1) th agent AHU at time t.
5 . The optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 1 , wherein training an actor network and a critic network of each agent specifically comprises:
allowing each agent i to interact with the environment at time t to obtain a state s i,t , and use the actor network to select and execute an action a i,t to obtain a new state s i,t+1 returned by the environment and a reward r i,t of each agent; storing experience (s i,t , a t , r t , s i,t+1 ) in an experience replay buffer, and randomly extracting a batch of experience samples from the experience replay buffer; calculating a target value for each agenti, minimizing the loss function, and updating parameters of the critic network; and updating parameters of the actor network through policy gradient for each agenti.
6 . The optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 5 , wherein for each agent i, parameters of a target network are softly updated:
ε
μ
i
′
←
τε
μ
i
+
(
1
-
τ
)
ε
μ
i
′
;
ε
Q
i
′
←
τε
Q
i
+
(
1
-
τ
)
ε
Q
i
′
;
wherein, τ is a soft update coefficient; ε represents a network weight parameter; μ i is an actor network of an i th agent,
μ
i
′
is a target actor network of the i th agent, Q i is a critic network of the i th agent, and
Q
i
′
is a target critic network of the i th agent; ε μ i represents an updated weight parameter of the actor network;
ε
μ
i
′
represents an updated weight parameter of the target actor network; ε Q i represents an updated weight parameter of the critic network;
ε
Q
i
′
represents an updated weight parameter of the target critic network.
7 . An optimized control system of building air conditionings taking personalized comfort of regional users into consideration, comprising:
a data acquisition module, being configured to: acquire comfort feedback data from different regional users of a building and historical operation data of a building air conditioning system, and perform data preprocessing; a Markov-decision-process construction module, being configured to: describe a heating, ventilation and air conditioning system control problem as a Markov decision process (MDP), and define each agent, and a state, an action and a reward of each agent; a meta-learning training module, being configure to: perform an initial parameter leaning of a comfort reward function for a plurality of regions based on the comfort feedback data from a plurality of regional users by using a meta-learning method to obtain a user comfort reward function initial model; a meta-learning testing module, being configured to: perform differentiated training and parameter updating on the user comfort reward function initial model by respectively using the comfort feedback data from the different regional users to obtain a differentiated comfort reward function model of each agent; and an optimized control policy training module, being configured to: obtain a reward function of each agent based on an energy consumption reward model and the differentiated comfort reward function model of each agent, training an actor network and a critic network of each agent, and making control decisions for the building heating, ventilation and air conditioning system by using the trained actor network; wherein: using the meta-learning method to obtain the user comfort reward function initial model specifically comprises: defining the comfort reward function of each regional user as a subtask; acquiring a plurality of sets of indoor environment states and comfort feedback data for the regional user that each subtask targets, and dividing into a Support set and a Query set; respectively training each subtask by using data in the Support set, and updating parameters of each subtask through gradient descent; performing testing in the Query set by using the parameters obtained through training, and then calculating a loss function under each subtask; averaging the loss functions of a plurality of subtasks, and updating an initial parameter of a meta-learning training task through gradient descent; calculating the loss function:
L
θ
,
b
=
1
K
∑
k
=
1
K
(
f
(
S
in
,
k
)
-
f
^
θ
(
S
in
,
k
)
)
2
;
wherein, ƒ(S in,k ) represents actual comfort feedback data from indoor users regarding indoor environment states obtained through sampling; {circumflex over (ƒ)} θ (S in,k ) represents a predicted output of a model represented by a weight parameter θ for the same input data S in,k ; K represents that the Support set totally contains K sets of data, and k represents a k(1≤k≤K) th set of data; S in,k represents a k th set of indoor environment data in the Support set; b represents a b th training subtask, 1≤b≤B;
updating parameters through gradient descent:
θ
′
=
θ
-
α
∇
θ
L
θ
,
b
;
wherein, θ′ is an updated weight parameter of a meta-learning network;
calculating the loss function under each subtask:
L
θ
′
,
b
=
1
K
′
∑
k
′
=
1
K
′
(
f
(
S
in
,
k
′
)
-
f
^
θ
″
(
S
in
,
k
′
)
)
2
;
wherein, S in,k ′ is a k′ th set of indoor environment data in the Query set; θ″ represents a model weight parameter obtained during testing by using the Query set after training with the Support set; K′ represents that the Query set totally contains K′ sets of data; k′ represents a k′ th (1≤k′≤K′) set of data; and
averaging the loss functions of all users, and updating an initial parameter through gradient descent:
θ
=
θ
-
β
·
1
B
∑
b
=
1
B
∇
θ
L
θ
′
,
b
;
wherein, b represents a b th training subtask, 1≤b≤B.
8 . A computer-readable storage medium, having a program stored thereon, wherein when the program is executed by a processor, implementing steps of the optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 1 .
9 . An electronic device, comprising a memory, a processor, and a program stored on the memory and runnable by the processor, wherein when the program is executed by the processor, implementing steps of the optimized control method of building air conditionings taking personalized comfort of regional users into consideration according to claim 1 .Join the waitlist — get patent alerts
Track US2026071771A1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.