Getting started

Solve finite horizon multi-stage deterministic decision problems

DynaProg provides a flexible tool to solve a finite horizon multi-stage deterministic decision problem, which is a problem where a decision must be made at each stage for a system that evolves through a finite number of stages, minimizing the total cost incurred.

This class of problems is described by a dynamic system and a cost function.

The dynamic system usually takes the form:

where

k indexes the stages (which may represent discrete time intervals or something more abstract),
characterizes the state of the system at stage k,
characterizes the control variables (the decision) to be taken at stage k,
N is the total number of stages.

The cost function takes the form:

J(x_0,u_0,...,u_(N-1)) = g_N(x_N) + SUM(from k=0 to N-1) of g(x_k, u_k),

where

is the stage cost, that is the cost incurred for each stage, and
is some terminal cost.

Obtaining the optimal solution of this optimization problem is therefore to obtain the minimum total cost

$J^*(x_0) = MIN over (u_k, k=0, ..., N-1) of J(x_0,u_0,...,u_{N-1})$

and the optimal control sequence u*_0, ..., u*_(N-1)

that minimizes it.

With DynaProg, you can model the dynamic system and a cost function as a MATLAB function in an m-file, and easily obtain the optimal solution, without having to develop your own implementation of a Dynamic Programming algorithm.

Additionally, you can specify constraints on the state and control variables and/or add an exogenous input which can influence the system dynamics.

Getting started

Set up a basic problem

Specify contraints on the state and control variables

Specify a terminal cost

Handle exogenous inputs

Use additional inputs

Use additional outputs

Splitting the model function

Troubleshooting your model and Safe Mode