Markov Chains and Stochastic Stability

This book describes the modern theory of general state space Markov chains, and the application of that theory to operations research, time series analysis, and systems and control theory.

**Tag(s):**
Operations Research

**Publication date**:

**ISBN-10**:
0387198326

**ISBN-13**:
n/a

**Paperback**:
548 pages

**Views**: 17,102

Markov Chains and Stochastic Stability

This book describes the modern theory of general state space Markov chains, and the application of that theory to operations research, time series analysis, and systems and control theory.

Book Excerpts:

This book describes the modern theory of general state space Markov chains, and the application of that theory to operations research, time series analysis, and systems and control theory. It is intended as an advanced graduate text in any of these areas, as well as being a research monograph incorporating a new and thorough treatment of the stability of general Markov chains.

There are several key themes in this book which interweave to a surprising extent in both the mathematics and its implementation. There is the use of the splitting technique, which provides an approach to general state space chains through regeneration methods; the systematic use of "Foster-Lyapunov" drift criteria, both in improving the theory and in enabling the classification of individual chains; the delineation of appropriate continuity conditions to link the general theory with the properties of chains on, in particular, Euclidean space; and the development of control model approaches, enabling analysis of models from their deterministic counterparts.

The applications cover storage systems, including some networks models as well as more traditional GI/G/1 queues and dam models; vector ARMA models including those with random coefficients and bilinear models; and both linear and non-linear state space systems with and without controls. To enhance accessibility, each chapter begins with a development of countable state space chains if appropriate. The general state space theory is then developed in close analogy, and where possible the theory is then specialized to chains on a topological state space, such as Euclidean space, so that the special structure of such spaces can be explored.

Book Overview:

The book is divided into three parts. Chains which are psi-irreducible are the focus of this book, and it is in Part I that this class of chains is initially developed. Numerous applications are described, and the foundations and basic definitions are developed directly, and illustrated through these applications. In the second part a description of the possible stable regimes for a Markov chain are described. The dichotomies which exist between transience and recurrence are developed here, and positive recurrence is described for general state space chains. Drift criteria are introduced to enable practical verification of the various forms of stability for specific models. In Part III existing and new ergodic theory is presented, and here the drift criterion approach makes its full impact. Total variation norm limit theorems, laws of large numbers, and functional central limit theorems are obtained through the combined use of drift criteria and splitting techniques.

Many of the theoretical results appear here for the first time, and much of the theory and the models which are used to illustrate the theory, and to provide extensions of the theory in special cases, have not previously been brought together in book form. This book thus provides a readable account of the development over the last two decades of a fundamental and applicable area of stochastic processes, and as such will be of value not only in probability theory but in the many discplines where these models form the basis of analysis.

This book describes the modern theory of general state space Markov chains, and the application of that theory to operations research, time series analysis, and systems and control theory. It is intended as an advanced graduate text in any of these areas, as well as being a research monograph incorporating a new and thorough treatment of the stability of general Markov chains.

There are several key themes in this book which interweave to a surprising extent in both the mathematics and its implementation. There is the use of the splitting technique, which provides an approach to general state space chains through regeneration methods; the systematic use of "Foster-Lyapunov" drift criteria, both in improving the theory and in enabling the classification of individual chains; the delineation of appropriate continuity conditions to link the general theory with the properties of chains on, in particular, Euclidean space; and the development of control model approaches, enabling analysis of models from their deterministic counterparts.

The applications cover storage systems, including some networks models as well as more traditional GI/G/1 queues and dam models; vector ARMA models including those with random coefficients and bilinear models; and both linear and non-linear state space systems with and without controls. To enhance accessibility, each chapter begins with a development of countable state space chains if appropriate. The general state space theory is then developed in close analogy, and where possible the theory is then specialized to chains on a topological state space, such as Euclidean space, so that the special structure of such spaces can be explored.

Book Overview:

The book is divided into three parts. Chains which are psi-irreducible are the focus of this book, and it is in Part I that this class of chains is initially developed. Numerous applications are described, and the foundations and basic definitions are developed directly, and illustrated through these applications. In the second part a description of the possible stable regimes for a Markov chain are described. The dichotomies which exist between transience and recurrence are developed here, and positive recurrence is described for general state space chains. Drift criteria are introduced to enable practical verification of the various forms of stability for specific models. In Part III existing and new ergodic theory is presented, and here the drift criterion approach makes its full impact. Total variation norm limit theorems, laws of large numbers, and functional central limit theorems are obtained through the combined use of drift criteria and splitting techniques.

Many of the theoretical results appear here for the first time, and much of the theory and the models which are used to illustrate the theory, and to provide extensions of the theory in special cases, have not previously been brought together in book form. This book thus provides a readable account of the development over the last two decades of a fundamental and applicable area of stochastic processes, and as such will be of value not only in probability theory but in the many discplines where these models form the basis of analysis.

Tweet

About The Author(s)

Professor and Robert C. Pittman Eminent Scholar Chair in the Department of Electrical and Computer Engineering at the University of Florida, the director of the Laboratory for Cognition & Control, and director of the Florida Institute for Sustainable Energy. Academic research interests include theory and applications of decision and control, stochastic processes, and optimization.

No information is available for this author.

Book Categories

Computer Science
16
Introduction to Computer Science
32
Introduction to Computer Programming
52
Algorithms and Data Structures
25
Artificial Intelligence
24
Computer Vision
30
Machine Learning
6
Neural Networks
22
Game Development and Multimedia
25
Data Communication and Networks
5
Coding Theory
16
Computer Security
9
Information Security
35
Cryptography
3
Information Theory
17
Computer Organization and Architecture
22
Operating Systems
2
Image Processing
11
Parallel Computing
4
Concurrent Programming
24
Relational Database
3
Document-oriented Database
14
Data Mining
16
Big Data
17
Data Science
23
Digital Libraries
22
Compiler Design and Construction
26
Functional Programming
11
Logic Programming
26
Object Oriented Programming
21
Formal Methods
70
Software Engineering
3
Agile Software Development
7
Information Systems
5
Geographic Information System (GIS)

Mathematics
68
Mathematics
14
Algebra
1
Abstract Algebra
27
Linear Algebra
3
Number Theory
8
Numerical Methods
2
Precalculus
10
Calculus
3
Differential Equations
5
Category Theory
11
Proofs
20
Discrete Mathematics
24
Theory of Computation
15
Graph Theory
2
Real Analysis
1
Complex Analysis
15
Probability
47
Statistics
7
Game Theory
5
Queueing Theory
13
Operations Research
16
Computer Aided Mathematics

Supporting Fields
21
Web Design and Development
1
Mobile App Design and Development
28
System Administration
2
Cloud Computing
10
Electric Circuits
6
Embedded System
28
Signal Processing
4
Network Science
3
Project Management

Operating System
Programming/Scripting
6
Ada
13
Assembly
34
C / C++
8
Common Lisp
2
Forth
35
Java
13
JavaScript
1
Lua
15
Microsoft .NET
1
Rexx
12
Perl
6
PHP
68
Python
12
R
1
Rebol
13
Ruby
2
Scheme
3
Tcl/Tk

Miscellaneous