Welcome to the fifth edition of PM Interview Prep Weekly! I'm Ajitesh, and this week we're diving into one of the most technical areas of product management: A/B testing.
Quick Thought: Generally, in real-life scenarios, A/B experiments are done to avoid any heavy investments and figure out early what might work for a specified goal. The solutions that you have mentioned are heavy tech investment ones. What are your thoughts on this?
Great point. In context of Meta this is a bit different:
1. They already have investment in AI + AR/VR, this is just testing in market. The investment decision has already been made and the tech to an extent do exists with them.
2. We're talking about 10s of Billions of impact so weeks packaging changes is likely worth it
For example, any algorithm changes in Google, including BERT, or any machine learning changes are heavily tested. A lot of investment goes in before hand.
I agree that doing big investment just for running A/B experiment defeats the purpose. But it made sense to me in this case.
Hey Ajitesh. Great Article.
Quick Thought: Generally, in real-life scenarios, A/B experiments are done to avoid any heavy investments and figure out early what might work for a specified goal. The solutions that you have mentioned are heavy tech investment ones. What are your thoughts on this?
Great point. In context of Meta this is a bit different:
1. They already have investment in AI + AR/VR, this is just testing in market. The investment decision has already been made and the tech to an extent do exists with them.
2. We're talking about 10s of Billions of impact so weeks packaging changes is likely worth it
For example, any algorithm changes in Google, including BERT, or any machine learning changes are heavily tested. A lot of investment goes in before hand.
I agree that doing big investment just for running A/B experiment defeats the purpose. But it made sense to me in this case.