Application of the algorithms

Post Reply
Xuan Mu
Posts: 1
Joined: Sat Jul 10, 2021 5:40 am

Application of the algorithms

Post by Xuan Mu »

Hi,

Your A/B testing courses are really interesting! I'm wondering if you can give us an example of how to use the available dataset. I understand that we cannot use real datasets as they are confidential to the companies, but maybe it is possible to use the available datasets on Kaggle. After learning the Thompson Sampling With Gaussian Reward lesson, I really want to know how to implement the code with actual data. In real-world business, practitioners must have both historical data and current data available for an A/B test. Could you give me an example of how the practitioners normally use these data in the A/B testing?
lazyprogrammer
Site Admin
Posts: 60
Joined: Sat Jul 28, 2018 3:46 am

Re: Application of the algorithms

Post by lazyprogrammer »

Good questions.

> I understand that we cannot use real datasets as they are confidential to the companies

That's just one part of the lecture (I believe it was mentioned in "Bandit Summary, Real Data, and Online Learning", but also may have been mentioned in others as well).

But if not, then reviewing "Bandit Summary, Real Data, and Online Learning" should be helpful.

The other part of the lecture is that "real data" in this case corresponds to "real users".

As mentioned, in order to get real users - you have to start your own website or advertisement service, or something like that.

It's not as easy as 1, 2, 3. Starting a business and acquiring "real users" is actually quite difficult.

So you have 2 choices:

Option 1) Start your own business, gain real users

Option 2) Join a company which already has real users


> but maybe it is possible to use the available datasets on Kaggle.

The dataset I included in the course is a real dataset.

However, note that the pertinent issue here is that the "data" is not a "dataset" - the algorithms run in "real-time" / "live" / "online".


> After learning the Thompson Sampling With Gaussian Reward lesson, I really want to know how to implement the code with actual data.

Ok, maybe review "Bandit Summary, Real Data, and Online Learning" for this as well.

Recall my rule "All data is the same". This means that "implementing the code" is exactly the same, no matter the data.

The "implementing the code" part would not change.


> In real-world business, practitioners must have both historical data and current data available for an A/B test.

In the context of bandit algorithms, this is not true.

As we have shown, it handles the explore-exploit dilemma and can start from scratch.
Post Reply

Return to “Bayesian Machine Learning in Python: A/B Testing”