We had some heated debates a while ago about what would happen if we opened up all of Khan Academy’s content for logged out users. Sal’s videos have always been open in this way, of course, but all the interactive exercises and statistical tracking and badges and stuff required an account.

It felt like the right move when we reconfirmed our belief that educational content should be as open and available as possible. We were also persuaded by Fred Wilson’s belief that giving logged out users more power is an effective way to empower your community. But we worried that registrations would drop, because by handing our 250+ exercises to logged out users, we’d be drastically shrinking the carrot on the other side of the “please login!” boundary.

We decided to go for it this summer. Figured I’d share some results.

A dog

4+ months after the change, we know a little more. Jace looked at the data and found:

• Registration rate hasn’t changed. Bit of a surprise for both sides of the debate.
• Percentage of our visitors who try an exercise problem increased by (relative) 10-15%.

So more visitors are trying exercises, but they’re not converting into more registered users…yet. We could have fun making up all sorts of explanations. Maybe we need to show off the badges and points unregistered users are accumulating a little bit more, maybe we’re not asking visitors to login forcefully enough, maybe doing math exercises makes people tired, maybe we don’t have enough pictures of golden retrievers on the login page.

Percentage of all visitors who use our exercises

Far more important than any random explanation is the fact that we’re now getting X% more data about how users learn (or don’t learn) thanks to these new exercise visitors. That data is powering some of the best work coming out of Khan Academy so far, so feeding more logged out users into our exercises is, as Jace puts it, a big win on data collection.

Plus, we’re confident that we could iterate and A/B test our way to higher registration rates for our new exercise users. It’s not our priority (for this week, at least). We’re busy cooking up important changes to the core learning experience.

*They don’t have access to everything. The ability to coach and communicate with other users is still restricted to users who have logged in.