"Good experiments advance product strategy. Bad experiments only advance metrics."

"Bad experiments only advance metrics."

"Good experiments are built around product themes."

"product team’s understanding of the customer and her needs."

"Good experiments drive impact by solving real user problems."

"Good experiments are about understanding true customer behavior around the things that matter."

"Good experiments are conceived as bets."

"You know they have a chance to fail, but based on the info you have available, it is a good investment to make."

"Bad experiments are endless optimizations."

"Businesses aren’t built on optimizations."

"Good experiments ask “What do we do next with what we learned?” They dig in on questions like why did it work or not work? Does this prove or disprove our hypothesis? Was it a good test of the hypothesis? If it failed, should we invest more to make it successful? If it succeeded, should we do more to make it even more successful?"

"Good experiments are used as a tool of humility. They are used to test something we believe in, but we are humble enough to know that we aren’t sure if it will work."

"Good experiments define success up-front. They properly instrument the metrics associated with the definition of success."

"Good experiments consider tradeoffs."

"Good experiments use tight exposure groups."

"Everyone in the test group experienced the change, with an equally sized true control group."

"Good experiments are communicated. They build a narrative around learning."

"Good experiments are integrated into the core product process. They incorporate cultural principles and think about long term team success."

"Good experiments have a plan for success."

"Bad experiments leave MVPs in the product forever."