US2023306505A1PendingUtilityA1
Extending finite rank deep kernel learning to forecasting over long time horizons
Est. expiryAug 5, 2039(~13 yrs left)· nominal 20-yr term from priority
G06N 3/09G06N 3/0499G06Q 40/02G06N 20/00G06N 3/084G06N 3/047G06N 3/08G06N 20/10
69
PatentIndex Score
0
Cited by
0
References
0
Claims
Abstract
In one embodiment a finite rank deep kernel learning method includes: receiving a training dataset; forming a plurality of training data subsets from the training dataset; for each respective training data subset of the plurality of training data subsets: calculating a subset-specific loss based on a loss function and the respective training data subset; and optimizing a model based on the subset-specific loss; determining a set of embeddings based on the optimized model; determining, based on the set of embeddings, a plurality of dot kernels; combining the plurality of dot kernels to form a composite kernel for a Gaussian process; receiving live data from an application; and predicting a plurality of values and a plurality of uncertainties associated with the plurality of values simultaneously using the composite kernel.
Claims
exact text as granted — not AI-modifiedWhat is claimed is:
1 . A finite rank deep kernel learning method, comprising:
receiving a training dataset; forming a plurality of training data subsets from the training dataset; for each respective training data subset of the plurality of training data subsets:
calculating a subset-specific loss based on a loss function and the respective training data subset; and
optimizing a model based on the subset-specific loss, wherein optimizing the model comprises minimizing the loss function, the loss function comprising
data
fit
loss
component
=
σ
-
2
y
2
2
-
∑
i
=
1
R
<
ϕ
i
(
X
)
,
y
>
2
σ
2
(
σ
2
+
ϕ
i
(
X
)
2
2
)
,
wherein:
σ represents uncertainty;
y represents a response variable;
i represents an index value;
R represents a rank;
ϕ i represents an embedding for an entity corresponding to index i; and
X represents input data;
determining a set of embeddings based on the optimized model;
determining, based on the set of embeddings, a plurality of dot kernels; and
combining the plurality of dot kernels to form a composite kernel for a Gaussian process.
2 . The method of claim 1 , further comprising:
receiving live data from an application; and predicting a plurality of values and a plurality of uncertainties associated with the plurality of values simultaneously using the composite kernel.
3 . The method of claim 2 , wherein predicting the plurality of values comprises determining a mean value of each prediction of the plurality of predictions.
4 . The method of claim 2 , wherein predicting the plurality of uncertainties comprises:
determining a covariance of each prediction of the plurality of predictions; and determining a variance of each prediction of the plurality of predictions.
5 . The method of claim 2 , wherein:
the live data comprises financial data, the application is a financial management application, the plurality of values comprises a plurality of predicted future financial transactions, and each uncertainty of the plurality of uncertainties associated with a respective predicted future financial transaction estimates a range of values of the respective predicted future transaction.
6 . The method of claim 2 , wherein:
the live data comprises resource utilization data, the application is a resource management application, the plurality of values comprises a plurality of predicted resources needs, and each uncertainty of the plurality of uncertainties associated with a respective predicted future resource need estimates a range of values of the respective resource need.
7 . The method of claim 2 , wherein:
the live data is user activity data, the application is a resource access control application, the plurality of values comprises a plurality of predicted user activities, and each uncertainty of the plurality of uncertainties associated with a respective predicted future user activity estimates a range of values of the respective user activity.
8 . The method of claim 1 , wherein the composite kernel for the Gaussian process is modeled as a linear combination of the plurality of dot kernels as: K(x,y)=Σ i=1 R ϕ i (x,ω)ϕ i (y,ω), wherein K(x, y) represents the composite kernel for input value x and response variable y, and ω represents one or more weights.
9 . The method of claim 1 , wherein the loss function further comprises a complexity component=Σ i=1 R log(σ 2 +∥ϕ i (X)∥ 2 2 )+(N−R)log σ 2 , wherein N represents a total number of entities.
10 . The method of claim 1 , wherein the loss function further comprises
a
regularity
loss
component
=
λσ
-
2
y
2
2
∑
i
<
j
<
ϕ
i
(
X
)
,
ϕ
j
(
X
)
>
2
ϕ
i
(
X
)
2
2
ϕ
j
(
X
)
2
2
,
wherein:
λ represents regularization; and
j represents a different index value.
11 . A system, comprising:
a memory comprising computer-executable instructions; one or more processors configured to execute the computer-executable instructions and cause the system to:
receive a training dataset;
form a plurality of training data subsets from the training dataset;
for each respective training data subset of the plurality of training data subsets:
calculate a subset-specific loss based on a loss function and the respective training data subset; and
optimize a model based on the subset-specific loss, wherein optimizing the model comprises minimizing the loss function, the loss function comprising a complexity component=Σ i=1 R log(σ 2 +∥ϕ i (X)∥ 2 2 )+(N−R)log σ 2 , wherein:
σ represents uncertainty;
i represents an index value;
R represents a rank;
ϕ i represents an embedding for an entity corresponding to index i;
X represents input data; and
N represents a total number of entities;
determine a set of embeddings based on the optimized model;
determine, based on the set of embeddings, a plurality of dot kernels; and
combine the plurality of dot kernels to form a composite kernel for a Gaussian process.
12 . The system of claim 11 , wherein the one or more processors are further configured to cause the system to:
receive live data from an application; and predict a plurality of values and a plurality of uncertainties associated with the plurality of values simultaneously using the composite kernel.
13 . The system of claim 12 , wherein in order to predict the plurality of values, the one or more processors are further configured to determine a mean value of each prediction of the plurality of predictions.
14 . The system of claim 12 , wherein in order to predict the plurality of uncertainties, the one or more processors are further configured to:
determine a covariance of each prediction of the plurality of predictions; and determine a variance of each prediction of the plurality of predictions.
15 . The system of claim 12 , wherein:
the live data comprises financial data, the application is a financial management application, the plurality of values comprises a plurality of predicted future financial transactions, and each uncertainty of the plurality of uncertainties associated with a respective predicted future financial transaction estimates a range of values of the respective predicted future transaction.
16 . The system of claim 12 , wherein:
the live data comprises resource utilization data, the application is a resource management application, the plurality of values comprises a plurality of predicted resources needs, and each uncertainty of the plurality of uncertainties associated with a respective predicted future resource need estimates a range of values of the respective resource need.
17 . The system of claim 12 , wherein:
the live data is user activity data, the application is a resource access control application, the plurality of values comprises a plurality of predicted user activities, and each uncertainty of the plurality of uncertainties associated with a respective predicted future user activity estimates a range of values of the respective user activity.
18 . The system of claim 12 , wherein the loss function further comprises
a
data
fit
loss
component
=
σ
-
2
y
2
2
-
∑
i
=
1
R
<
ϕ
i
(
X
)
,
y
>
2
σ
2
(
σ
2
+
ϕ
i
(
X
)
2
2
)
,
wherein y represents a response variable.
19 . The system of claim 11 , wherein the loss function further comprises
a
regularity
loss
component
=
λσ
-
2
y
2
2
∑
i
<
j
<
ϕ
i
(
X
)
,
ϕ
j
(
X
)
>
2
ϕ
i
(
X
)
2
2
ϕ
j
(
X
)
2
2
,
wherein:
λ represents regularization; and
j represents a different index value;
y represents a response variable.
20 . A non-transitory computer readable medium comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to:
receive a training dataset; form a plurality of training data subsets from the training dataset; for each respective training data subset of the plurality of training data subsets:
calculate a subset-specific loss based on a loss function and the respective training data subset; and
optimize a model based on the subset-specific loss, wherein optimizing the model comprises minimizing the loss function, the loss function comprising
a
regularity
loss
component
=
λσ
-
2
y
2
2
∑
i
<
j
<
ϕ
i
(
X
)
,
ϕ
j
(
X
)
>
2
ϕ
i
(
X
)
2
2
ϕ
j
(
X
)
2
2
,
wherein:
λ represents regularization; and
σ represents uncertainty;
i represents an index value;
j represents a different index value;
y represents a response variable;
ϕ i represents an embedding for an entity corresponding to index i; and
X represents input data;
determine a set of embeddings based on the optimized model;
determine, based on the set of embeddings, a plurality of dot kernels; and
combine the plurality of dot kernels to form a composite kernel for a Gaussian process.Join the waitlist — get patent alerts
Track US2023306505A1 — get alerts on status changes and closely related new filings.
We store only your email — no account needed. See our privacy policy.