I reviewed the material on Non Stationary Bandits - and I understand the formula below works is a running calculation of the exponential weighted average favoring recent data:

new_mean = (1- alpha) * old_mean + alpha * x

I was trying to understand - how I could adapt this formula to perform a running calculation for the parameters - "a" and "b" of the beta distribution.

I haven't been able to figure this out yet. The closest I could get was to maintain a buffer containing the last N rewards - and then use that to estimate "a" and "b".

The problem with this approach is that my posterior is always roughly the same width - it doesn't get "skinnier" or "fatter".

Ideally, If the rewards of my bandit are changing over time - I was hoping the posteriors would shift from "skinny" (old stable click-thru rate) to "fat" (period of uncertainty) and then finally "skinny" again (new stable click-thru rate).

It would be great if the training material could touch on this subject.

BTW - I just hope I didn't miss it if you already covered this in the training.

## Non Stationary Bandits

Return to “Bayesian Machine Learning in Python: A/B Testing”

Jump to

- General Discussion
- ↳ General Discussion
- Course Q&As
- ↳ Deep Learning Prerequisites: The Numpy Stack in Python
- ↳ Deep Learning Prerequisites: Linear Regression in Python
- ↳ Deep Learning Prerequisites: Logistic Regression in Python
- ↳ Data Science: Deep Learning in Python
- ↳ Modern Deep Learning in Python
- ↳ Deep Learning: Convolutional Neural Networks in Python
- ↳ Unsupervised Deep Learning in Python
- ↳ Deep Learning: Recurrent Neural Networks in Python
- ↳ Data Analytics: SQL for newbs, beginners and marketers
- ↳ Data Science: Natural Language Processing (NLP) in Python
- ↳ Natural Language Processing with Deep Learning in Python
- ↳ Data Science: Supervised Machine Learning in Python
- ↳ Cluster Analysis and Unsupervised Machine Learning in Python
- ↳ Bayesian Machine Learning in Python: A/B Testing
- ↳ Unsupervised Machine Learning Hidden Markov Models in Python
- ↳ Ensemble Machine Learning in Python: Random Forest, AdaBoost
- ↳ Artificial Intelligence: Reinforcement Learning in Python
- ↳ Advanced AI: Deep Reinforcement Learning in Python
- ↳ Deep Learning: GANs and Variational Autoencoders
- ↳ Deep Learning: Advanced Computer Vision (Computer Vision Applications with Deep Learning)
- ↳ Deep Learning: Advanced NLP and RNNs (Advanced NLP and Sequence Models with Deep Learning)
- ↳ Recommender Systems and Deep Learning in Python (Recommender System Applications with Deep Learning)
- ↳ Machine Learning and AI: Support Vector Machines in Python
- ↳ Cutting-Edge AI: Deep Reinforcement Learning in Python
- ↳ MATLAB for Students, Engineers, and Professionals in STEM
- ↳ Tensorflow 2.0: Deep Learning and Artificial Intelligence
- ↳ PyTorch: Deep Learning and Artificial Intelligence
- ↳ Linear Programming for Linear Regression
- ↳ Classical Statistical Inference