To learn more about SRM and how to resolve it, see the Sample Ratio Mismatch section in Experimentation Best Practices and Recommendations
- Player count, Average, Delta, Statistical significance, P-value, P-move, and Stanard deviation metrics are produced
- If an issue of Sample Ratio Mismatch (SRM) is identified, then it’s flagged to the user for investigation.
- SRM indicates sampling bias in the randomization, which leads to the options of resolving bugs (if any), immediately stop the experiment based on the estimated impact or take the risk and let the experiment continue.
- The overall statistical computation of each metric is available for analysis.
