Class Reference¶

The main class of the dynamic linear model.

This is the main class of the Bayeisan dynamic linear model. It provides the modeling, filtering, forecasting and smoothing function of a dlm. The dlm use the @builder to construct the @baseModel based on user supplied @components and then run @kalmanFilter to filter the result.

Example 1:

>>> # randomly generate fake data on 1000 days
>>> import numpy as np
>>> data = np.random.random((1, 1000))
>>> # construct the dlm of a linear trend and a 7-day seasonality
>>> myDlm = dlm(data) + trend(degree = 2, 0.98) + seasonality(period = 7, 0.98)
>>> # filter the result
>>> myDlm.fitForwardFilter()
>>> # extract the filtered result
>>> myDlm.getFilteredObs()

Example 2 (fit a linear regression):

>>> from pydlm import dynamic
>>> data = np.random.random((1, 100))
>>> mydlm = dlm(data) + trend(degree=1, 0.98, name='a') +
                dynamic(features=[[i] for i in range(100)], 1, name='b')
>>> mydlm.fit()
>>> coef_a = mydlm.getLatentState('a')
>>> coef_b = mydlm.getLatentState('b')

data¶

a list of doubles of the raw time series data. It could be either the python’s built-in list of doubles or numpy 1d array.

add(component)¶

Add new modeling component to the dlm.

Currently support: trend, seasonality, autoregression and dynamic component.

Parameters:	component – the modeling component, could be either one of the following: trend, seasonality, dynamic, autoReg.
Returns:	A dlm object with added component.

alter(date, data, component='main')¶

To alter the data for a specific date and a specific component.

Parameters:	date – the date of the altering data data – the new data. data must be a numeric value for main time series and must be a list of numerical values for dynamic components. component – the component for which the new data need to be supplied to. ‘main’: the main time series data other component name: other component feature data

append(data, component='main')¶

Append the new data to the main data or the components (new feature data)

Parameters:	data – the new data component – the name of which the new data to be added to. ‘main’: the main time series data other omponent name: add new feature data to other component.

continuePredict(featureDict=None)¶

Continue prediction after the one-day ahead predict.

If users want to have a multiple day prediction, they can opt to use continuePredict after predict with new features contained in featureDict. For example,

>>> # predict 3 days after the last day
>>> myDLM.predict(featureDict=featureDict_day1)
>>> myDLM.continuePredict(featureDict=featureDict_day2)
>>> myDLM.continuePredict(featureDict=featureDict_day3)

The featureDict acts the same way as in predict().

Parameters:	featureDict – the feature set for the dynamic components, stored in a for of {“component name”: vector}. If the set was not supplied, then the algo will re-use the old feature. For days beyond the data, the featureDict for every dynamic component must be provided.
Returns:	A tupe. (predicted observation, variance)

delete(name)¶

Delete model component by its name

Parameters:	name – the name of the component.

evolveMode(evoType='dependent')¶

Control whether different component evolve indpendently. If true, then the innovation will only be added on each component but not the correlation between the components, so that for component with discount equals to 1, the smoothed results will always be constant.

Parameters:	evoType – If set to ‘independent’, then each component will evolve independently. If set to ‘dependent’, then the components will proceed jointly. Default to ‘independent’. Switch to ‘dependent’ if efficiency is a concern.
Returns:	a dlm object (for chaining purpose)

fit()¶

An easy caller for fitting both the forward filter and backward smoother.

fitBackwardSmoother(backLength=None)¶

Fit backward smoothing on the data. Starting from the last observed date.

Parameters:	backLength – integer, indicating how many days the backward smoother go, starting from the last date. (should) –

fitForwardFilter(useRollingWindow=False, windowLength=3)¶

Fit forward filter on the available data.

User can choose use rolling windowFront or not. If user choose not to use the rolling window, then the filtering will be based on all the previous data. If rolling window is used, then the filtering for a particular date will only consider previous dates that are within the rolling window length.

Parameters:	useRollingWindow – indicate whether rolling window should be used. windowLength – the length of the rolling window if used.

getAll()¶

get all the _result class which contains all results

Returns:	The @result object containing all computed results.

getInterval(p=0.95, filterType='forwardFilter', name='main')¶

get the confidence interval for data or component.

If the filtered dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	p – The confidence level. filterType – the type of CI to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’. name – the component to get CI. When name = ‘main’, then it returns the confidence interval for the time series. When name = some component’s name, then it returns the confidence interval for that component. Default to ‘main’.
Returns:	A tuple with the first element being a list of upper bounds and the second being a list of the lower bounds.

getLatentCov(filterType='forwardFilter', name='all')¶

get the error covariance for different components and filters.

If the filtered dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	filterType – the type of latent covariance to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’. name – the component to get latent cov. When name = ‘all’, then it returns the latent covariance for the time series. When name = some component’s name, then it returns the latent covariance for that component. Default to ‘all’.
Returns:	A list of numpy matrices, standing for the filtered latent covariance.

getLatentState(filterType='forwardFilter', name='all')¶

get the latent states for different components and filters.

If the filtered dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	filterType – the type of latent states to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’. name – the component to get latent state. When name = ‘all’, then it returns the latent states for the time series. When name = some component’s name, then it returns the latent states for that component. Default to ‘all’.
Returns:	A list of lists, standing for the latent states given the different choices.

getMSE()¶

Get the one-day ahead prediction mean square error. The mse is estimated only for days that has been predicted.

Returns:	An numerical value

getMean(filterType='forwardFilter', name='main')¶

get mean for data or component.

If the working dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	filterType – the type of mean to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’. name – the component to get mean. When name = ‘main’, then it returns the filtered mean for the time series. When name = some component’s name, then it returns the filtered mean for that component. Default to ‘main’.
Returns:	A list of the time series observations based on the choice

getResidual(filterType='forwardFilter')¶

get the residuals for data after filtering or smoothing.

If the working dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	filterType – the type of residuals to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’.
Returns:	A list of residuals based on the choice

getVar(filterType='forwardFilter', name='main')¶

get the variance for data or component.

If the filtered dates are not (0, self.n - 1), then a warning will prompt stating the actual filtered dates.

Parameters:	filterType – the type of variance to be returned. Could be ‘forwardFilter’, ‘backwardSmoother’, and ‘predict’. Default to ‘forwardFilter’. name – the component to get variance. When name = ‘main’, then it returns the filtered variance for the time series. When name = some component’s name, then it returns the filtered variance for that component. Default to ‘main’.
Returns:	A list of the filtered variances based on the choice.

ignore(date)¶

Ignore the data for a specific day. treat it as missing data

Parameters:	date – the date to ignore.

ls()¶

List out all existing components

noisePrior(prior=0)¶

To set the prior for the observational noise. Calling with empty argument will enable the auto noise intializer (currently, the min of 1 and the variance of time series).

Parameters:	prior – the prior of the observational noise.
Returns:	A dlm object (for chaining purpose)

plot(name='main')¶

The main plot function. The dlmPlot and the matplotlib will only be loaded when necessary.

Parameters:	name – component to plot. Default to ‘main’, in which we plot the filtered time series. If a component name is given It plots the mean of the component, i.e., the observed value that attributes to that particular component, which equals to evaluation * latent states for that particular component.

plotCoef(name, dimensions=None)¶

Plot function for the latent states (coefficents of dynamic component).

Parameters:	name – the name of the component to plot. It plots the latent states for the component. If dimension of the given component is too high, we truncate to the first five. Or the user can supply the ideal dimensions for plot in the dimensions parameter. dimensions – dimensions will be used as the indexes to plot within that component latent states.

plotPredictN(N=1, date=None, featureDict=None)¶

Function to plot the N-day prediction results.

The input is the same as dlm.predictN. For details, please refer to that function.

Parameters:

N – The length of days to predict. date – The index when the prediction based on. Default to the last day. FeatureDict – The feature set for the dynamic Components, in a form of {“component_name”: feature}, where the feature must have N elements of feature vectors. If the featureDict is not supplied, then the algo reuses those stored in the dynamic components. For dates beyond the last day, featureDict must be supplied.

popout(date)¶

Pop out the data for a given date

Parameters:	date – the index indicates which date to be popped out.

predict(date=None, featureDict=None)¶

One day ahead predict based on the current data.

The predict result is based on all the data before date and predict the observation at date + days.

The prediction could be on the last day and into the future or in the middle of the time series and ignore the rest. For predicting into the future, the new features must be supplied to featureDict. For prediction in the middle, the user can still supply the features which will be used priorily. The old features will be used if featureDict is None.

Parameters:	date – the index when the prediction based on. Default to the last day. featureDict – the feature set for the dynamic Components, in a form of {“component_name”: feature}. If the featureDict is not supplied, then the algo reuse those stored in the dynamic components. For dates beyond the last day, featureDict must be supplied.
Returns:	A tuple. (Predicted observation, variance of the predicted observation)

predictN(N=1, date=None, featureDict=None)¶

N day ahead prediction based on the current data.

This function is a convenient wrapper of predict() and continuePredict(). If the prediction is into the future, i.e, > n, the featureDict has to contain all feature vectors for multiple days for each dynamic component. For example, assume myDLM has a component named ‘spy’ which posseses two dimensions,

>>> featureDict_3day = {'spy': [[1, 2],[2, 3],[3, 4]]}
>>> myDLM.predictN(N=3, featureDict=featureDict_3day)

Parameters:

N – The length of days to predict. date – The index when the prediction based on. Default to the last day. FeatureDict – The feature set for the dynamic Components, in a form of {“component_name”: feature}, where the feature must have N elements of feature vectors. If the featureDict is not supplied, then the algo reuse those stored in the dynamic components. For dates beyond the last day, featureDict must be supplied.

Returns:

A tuple of two lists. (Predicted observation, variance of the predicted observation)

resetPlotOptions()¶

Reset the plotting option for the dlm class

setColor(switch, color)¶

“set” Operation for the dlm plotting colors

Parameters:	switch – key word. Controls over filtered/smoothed/predicted results, color – the color for the corresponding keyword.

setConfidence(p=0.95)¶

Set the confidence interval for the plot

setIntervalType(intervalType)¶

Set the confidence interval type

showOptions()¶

Print out all the option values

stableMode(use=True)¶

Turn on the stable mode, i.e., using the renewal strategy.

Indicate whether the renew strategy should be used to add numerical stability. When the filter goes over certain steps, the information contribution of the previous data has decayed to minimum. In the stable mode, We then ignore those days and refit the time series starting from current - renewTerm, where renewTerm is computed according to the discount. Thus, the effective sample size of the dlm is twice renewTerm. When discount = 1, there will be no renewTerm, since all the information will be passed along.

tune(maxit=100)¶

Automatic tuning of the discounting factors.

The method will call the model tuner class to use the default parameters to tune the discounting factors and change the discount factor permenantly. User needs to refit the model after tuning.

If user wants a more refined tuning and not change any property of the existing model, they should opt to use the @modelTuner class.

turnOff(switch)¶

“turn off” Operation for the dlm plotting options.

Parameters:

switch – The key word to switch off. ‘filtered plot’, ‘filter’ to not plot filtered results ‘smoothed plot’, ‘smooth’ to not plot smoothed results ‘predict plot’, ‘predict’, to not plot one-step ahead results ‘confidence interval’, ‘confierence’, ‘CI’ to not plot CI’s ‘data points’, ‘data’, ‘data point’ to not plot original data ‘multiple’, ‘separate’ to not plot results in separate figures ‘fitted dots’, ‘fitted’ to not plot fitted results with dots

turnOn(switch)¶

“turn on” Operation for the dlm plotting options.

Parameters:

switch – The key word to switch on. ‘filtered plot’, ‘filter’ to plot filtered results ‘smoothed plot’, ‘smooth’ to plot smoothed results ‘predict plot’, ‘predict’, to plot one-step ahead results ‘confidence interval’, ‘confierence’, ‘CI’ to plot CI’s ‘data points’, ‘data’, ‘data point’ to plot original data ‘multiple’, ‘separate’ to plot results in separate figures ‘fitted dots’, ‘fitted’ to plot fitted results with dots

The trend component that features the polynomial trending, providing one building block for the dynamic linear model. It decribes a latent polynomial trending in the time series data.

Examples

>>>  # create a constant trend
>>> ctrend = trend(degree = 1, name = 'Const', discount = 0.99)
>>>  # change the ctrend to have covariance with diagonals are 2 and state 1
>>> ctrend.createCovPrior(cov = 2)
>>> ctrend.createMeanPrior(mean = 1)

d¶

the dimension of the latent states of the polynomial trend

componentType¶

the type of the component, in this case, ‘trend’

name¶

the name of the trend component, to be supplied by user used in modeling and result extraction

discount¶

the discount factor for this component. Details please refer to the @kalmanFilter

evaluation¶

the evaluation matrix for this component

transition¶

the transition matrix for this component

covPrior¶

the prior guess of the covariance matrix of the latent states

meanPrior¶

the prior guess of the latent states

checkDimensions()¶

if user supplies their own covPrior and meanPrior, this can be used to check if the dimension matches

createCovPrior(cov=10000000.0)¶

Create the prior covariance matrix for the latent states.

createEvaluation()¶

Create the evaluation matrix

createMeanPrior(mean=0)¶

Create the prior latent state

createTransition()¶

Create the transition matrix

According Hurrison and West (1999), the transition matrix of trend takes a form of

[[1 1 1 1],

[0 1 1 1],

[0 0 1 1],

[0 0 0 1]]

The seasonality component that features the periodicity behavior, providing one building block for the dynamic linear model. It decribes a latent seasonality trending in the time series data. The user can use this class to construct any periodicy component to the model, for instance, the hourly, weekly or monthly behavior. Different from the Fourier series, the seasonality components are nonparametric, i.e., there is no sin or cos relationship between each state. They can be arbitrarily valued.

Examples

>>>  # create a 7-day seasonality:
>>> ctrend = seasonality(period = 7, name = 'weekly', discount = 0.99)
>>>  # change the ctrend to have covariance with diagonals are 2 and state 1
>>> ctrend.createCovPrior(cov = 2)
>>> ctrend.createMeanPrior(mean = 1)
>>> ctrend.freeForm()

d¶

the period of the seasonality

componentType¶

the type of the component, in this case, ‘seasonality’

name¶

the name of the seasonality component, to be supplied by user used in modeling and result extraction

discount¶

the discount factor for this component. Details please refer to the @kalmanFilter

evaluation¶

the evaluation matrix for this component

transition¶

the transition matrix for this component

covPrior¶

the prior guess of the covariance matrix of the latent states

meanPrior¶

the prior guess of the latent states

checkDimensions()¶

if user supplies their own covPrior and meanPrior, this can be used to check if the dimension matches

createCovPrior(cov=10000000.0)¶

Create the prior covariance matrix for the latent states.

createEvaluation()¶

Create the evaluation matrix

createMeanPrior(mean=0)¶

Create the prior latent state

createTransition()¶

Create the transition matrix.

According to Hurrison and West (1999), the transition matrix of seasonality takes a form of

[[0 1 0 0],

[0 0 1 0],

[0 0 0 1],

[1 0 0 0]]

freeForm()¶

The technique used in Hurrison and West (1999). After calling this method, The latent states sum up to 0 and the covariance matrix is degenerate to have rank d - 1, so that the sum of the latent states will never change when the system evolves

The dynamic component that allows user to add controlled variables, providing one building block for the dynamic linear model. It decribes a dynamic component in the time series data. It basically allows user to supply covariate or controlled variable into the dlm, and the coefficients of the features will be trained as the latent states. Examples are holiday indicators, other observed variables and so on.

Examples

>>>  # create a dynamic component:
>>> features = [[1.0, 2.0] for i in range(10)]
>>> ctrend = dynamic(features = features, name = 'random', discount = 0.99)
>>>  # change the ctrend to have covariance with diagonals are 2 and state 1
>>> ctrend.createCovPrior(cov = 2)
>>> ctrend.createMeanPrior(mean = 1)

d¶

the dimension of the features (number of latent states)

n¶

the number of observation

componentType¶

the type of the component, in this case, ‘dynamic’

name¶

the name of the trend component, to be supplied by user used in modeling and result extraction

discount¶

the discount factor for this component. Details please refer to the @kalmanFilter

evaluation¶

the evaluation matrix for this component

transition¶

the transition matrix for this component

covPrior¶

the prior guess of the covariance matrix of the latent states

meanPrior¶

the prior guess of the latent states

alter(date, feature)¶

Change the corresponding

feature matrix.

Parameters:	date – The date to be modified. dataPoint – The new feature to be filled in.

appendNewData(newData)¶

For updating feature matrix when new data is added.

Parameters:	newData – is a list of list. The inner list is the feature vector. The outer list may contain multiple feature vectors.

checkDimensions()¶

if user supplies their own covPrior and meanPrior, this can be used to check if the dimension matches

createCovPrior(cov=None, scale=1000000.0)¶

Create the prior covariance matrix for the latent states

createEvaluation(step)¶

The evaluation matrix for the dynamic component change over time. It equals to the value of the features or the controlled variables at a given date

createMeanPrior(mean=None, scale=1)¶

Create the prior latent state

createTransition()¶

Create the transition matrix.

For the dynamic component, the transition matrix is just the identity matrix

hasMissingData(features)¶

Check whether the list contains None

popout(date)¶

For deleting the feature data of a specific date.

Parameters:	date – the index of which to be deleted.

updateEvaluation(step)¶

update the evaluation matrix to a specific date This function is used when fitting the forward filter and backward smoother in need of updating the correct evaluation matrix

The autoReg class allows user to add an autoregressive component to the dlm. This code implements the autoregressive component as a child class of component. Different from the dynamic component, the features in the autoReg is generated from the data, and updated according to the data.

The latent states of autoReg are aligned in the order of [today - degree, today - degree + 1, ..., today - 2, today - 1]. Thus, when fetching the latents from autoReg component, use this order to correctly align the coefficients.

Examples

>>>  # create a auto regression component:
>>> autoReg8 = autoReg(degree=8, name='autoreg8', discount = 0.99)
>>>  # change the autoReg8 to have covariance with diagonals are 2 and state 1
>>> autoReg8.createCovPrior(cov = 2)
>>> autoReg8.createMeanPrior(mean = 1)

d¶

the degree of autoregressive, i.e., how many days to look back

data¶

deprecatd – Users get a warning if this argument is used.

discount factor

the discounting factor

name¶

the name of the component

padding¶

either 0 or None. The number to be padded for the first degree days, as no previous data is observed to form the feature matrix

evaluation¶

the evaluation matrix for this component

transition¶

the transition matrix for this component

covPrior¶

the prior guess of the covariance matrix of the latent states

meanPrior¶

the prior guess of the latent states

checkDimensions()¶

if user supplies their own covPrior and meanPrior, this can be used to check if the dimension matches

createCovPrior(cov=None, scale=1000000.0)¶

Create the prior covariance matrix for the latent states

createEvaluation(step, data)¶

The evaluation matrix for auto regressor.

createMeanPrior(mean=None, scale=1)¶

Create the prior latent state

createTransition()¶

Create the transition matrix.

For the auto regressor component, the transition matrix is just the identity matrix

The longSeason class alows user to add a long seasonality component to the dlm. The difference between the long seasonality is that 1) The seasonality component use each date as a unit and change in a given periodicity. For example, 1, 2, 3, 4, 1, 2, 3, 4. 2) However, the long seasonality is capable to group couple of days as the basic unit and change in a periodicity. For example, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4. The usecase for long seasonality is to model longer seasonality with the short-term seasonality. For example, the short-term seansonality can be used to model the day of a weak patten and the long seasonality can be used to model the week of a month patten in the same model. This code implements the longSeason component as a sub-class of dynamic. Different from the dynamic component, the features in the autoReg is generated from the data, and updated according to the data. All other features are similar to @dynamic.

period¶

the periodicity, i.e., how many different states it has in one period

stay¶

the length of a state last.

discount factor

the discounting factor

name¶

the name of the component

alter(date, dataPoint)¶

We do nothing to longSeason, when altering the main data

appendNewData(newData)¶

Append new data to the existing features. Overriding the same method in @dynamic

Parameters:	newData – a list of new data

checkDataLength()¶

Check whether the degree is less than the time series length

createFeatureMatrix(period, stay, n, state)¶

Create the feature matrix based on the supplied data and the degree.

Parameters:	period – the periodicity of the component stay – the length of the base unit, i.e, how long before change to change to the next state.

popout(date)¶

Pop out the data of a specific date and rewrite the correct feature matrix.

Parameters:	date – the index of which to be deleted.

updateEvaluation(step)¶

update the evaluation matrix to a specific date This function is used when fitting the forward filter and backward smoother in need of updating the correct evaluation matrix

The main class for modelTuner

method¶

the optimization method. Currently only ‘gradient_descent’ is supported.

loss¶

the optimization loss function. Currently only ‘mse’ (one-day ahead prediction) is supported.

getDiscounts()¶

Get the tuned discounting factors. One for each component (even the component being multi-dimensional, only one discounting factor will be assigned to one component). Initialized to None.

tune(untunedDLM, maxit=100, step=1.0)¶

Main function for tuning the DLM model.

Parameters:	untunedDLM – The DLM object that needs tuning maxit – The maximum number of iteractions for gradient descent. step – the moving length at each iteraction.
Returns:	A tuned DLM object in unintialized status.