The 5th biggest site on the internet runs without ads, and without government support. That site is Wikipedia (hosted by the Wikimedia Foundation), and it survives on small-dollar donations! Those donations serve 500 million users, keep servers running, and power the entire organization.
Wikipedia and its small-donor funding model is something quite special. We use testing and experimentation to figure out the best way to explain the Wikimedia story to our users, and improve our fundraising efficiency. While doing so, we hope to cause minimum disruption to any single visitor. Testing and analysis is helping us reach that goal.
We've created this platform to showcase some of our significant tests and findings.
Right now, the Wikimedia fundraising team is in the middle of a systematic review of all our prior tests. Although we've posted many examples of our tests over the years, this review allows us to publish them in a standard format. We will publish as many results here as we can.
Today, we're showcasing our first batch of about 50 tests. Each shows a strong and statistically significant result. The first 36 show the evolution of our banners over time. (Though they're not all strictly chronological). The latter tests are in no particular order, because each was quite similar to one of those first tests. Don't worry, you'll know when we move from the first set of tests to the next.
There are a few different reasons why we conduct similar tests: to understand better what's making a difference; to try out something that seems to be working in different languages, countries, times of day, etc; and just to make sure that our results were accurate.
Over time, we hope to release and present many more tests. These tests are by no means the only significant results we've found - just a small sample. This batch only includes simple banner tests.
We've decided to show our results in an interactive way. You'll be presented with two different variations. For example, a gold background vs a blue background for a banner. After guessing which banner "won", you'll see the correct answer, and a link to some of the internal diagnostics we use to check for anomalies.
For the purposes of this presentation, the winning banner is the one which got the most donations per banner impression. Though we're purposefully ignoring the donation amount in this presentation, we do take it into account when doing the actual testing. In the future, we'll release analysis of the donation amounts as well.
What we tested:
Nearly all of our donations come from running banners at the top of the page. The banner changes as our testing uncovers better-performing variations. And we're testing all the time.
Though we're focusing on banners in this release, we've tested other variables as well. We've spent hundreds of tests on:
- Appeal Letters
- Donation Form Design
- Payment Methods
- Banner/Landing page combinations
- Other variables
Why this is important:
To quote from our report, How Wikimedia Grows,
Wikimedia projects are funded by an incredibly unusual revenue model. We are showing that a top website, a universal educational resource, a central pillar of the world's information infrastructure, can thrive without selling its soul and independence to private investors or advertisers. This revenue model is one of the things that ensures our ability to serve our sites for free with all content under a free license.
Wikimedia's mission is to bring free educational content to the world. Part of that education involves sharing the knowledge we've learned through billions of pageviews worth of testing on the world's 5th biggest website.
We hope you enjoy this look into our testing process.