May 2012
2 posts
App Engine Performance Essentials You'll Certainly...
It’s kinda cheating to call this Part II of App Engine Performace Hacks You’ll Probably Never Need, but I recently stumbled across a single tip that anybody in App Engine land will want to know about.
Don’t use fetch() for datastore access, it’s slower than it used to be
If you’ve been on App Engine for a while, you’re either surprised by that tip or more...
How intern mentorship works at Khan Academy
Three of our twelve(‽) summer interns have arrived. It’s high time I share how we mentor interns before I’m spending all my time swatting sharks and helicopters out of the air. These tips also apply to new full-time hires, of course, but for now I’ve got interns on the brain.
1. Make the mentorship relationship explicit
At some point in a new employee’s first day they...
April 2012
1 post
"Step one? FIX. Step two? IT."
“Step three? FIXIT. Now repeat till it is FIXED.”
Wise words from Kenan, patron saint of the first Khan Academy fixit.
Fixit day is just one more in the long list of solid dev lessons I’ve been learning from the Googlers around here. Since I couldn’t find a reputable source explaining the fixit culture (who reads the NY Times?), I figured it’s my duty to share.
...
March 2012
1 post
Teacher effectiveness ratings, programmers, and...
Today I was doing whatever it is I do when I ran across this link from Joel:
NYC teacher “effectiveness” ratings are bogus, and the data prove it garyrubinstein.teachforus.org/2012/02/28/ana…— Joel Spolsky (@spolsky) March 1, 2012
…and my brain started pattern matching. Replace “NYC teacher” with “programmer” in this tweet, and you’d be in...
February 2012
1 post
Required code reviews
This is the story of the growing Khan Academy team converting me into a passionate fan of requiring a code review for every single changeset.
Those who have worked with me know that it’s a surprising position for me to take. On the spectrum of “Follows good development practices even if it slows down the product” to “Just ship the thing, code doesn’t matter, only...
January 2012
2 posts
Sharing the inspiring, personal stories of Khan...
We have these emails hanging up all over our office, sent in from Khan Academy users with incredible, personal stories to tell. Every time I read a new one I’m emotionally affected, which means my robot emotion chip is faulty.
So when some curious soul (like a reporter) wanders in and asks me, “How will you know if Khan Academy is really successful?” I always answer their...
Khan Academy Internship, Fall '11
I already can’t wait to drop some major challenges in the laps of our two incoming Fall interns to see what they can build.
- Khan Academy Internship, Summer ‘11
Check!
David Hu and Julian Pulgarin stepped up to the plate this Fall during their coopsinternships from University of Waterloo. We call ‘em internships because we’re from Amurrrrica, you crazy Canadians.
When...
December 2011
3 posts
.end() makes jQuery DOM traversal beautiful
This is my new favorite jQuery trick. I just learned it this year and have mentioned it in enough code reviews to decide it’s worth sharing.
When manipulating the DOM with jQuery, you often see code that looks something like:
$("#container").show();
$("#container .error").hide();
$("#container .zoo").css("background-color", "white");
$("#container .zoo...
Laughing at Others' Code
Those who have worked with me will know that I’m an expert on this topic because my code gets laughed at all the time.
I’ve seen Good laughing and Bad laughing. Good is what I imagine happens when Robert De Niro sits next to Al Pacino as they’re watching Al’s cameo in some trainwreck of an Adam Sandler movie, and Bob turns to Al smiling and says, “This is...
After giving logged out users access to pretty...
We had some heated debates a while ago about what would happen if we opened up all of Khan Academy’s content for logged out users. Sal’s videos have always been open in this way, of course, but all the interactive exercises and statistical tracking and badges and stuff required an account.
It felt like the right move when we reconfirmed our belief that educational content should be...
November 2011
2 posts
How to make "consecutive days of Khan Academy...
Think how cool it would be if users were rewarded after learning for X consecutive days.
Think about implementation for a second and then go, “Sweet! This’ll be trivial Let’s knock it out Here we go I’ve got things to do!”
Write a line of code or two.
Get sucker-punched by the gruesome specter of timezones when realizing that a student doing work at 8am on...
A/B testing still works. [Sarcastic *PHEW*].
After releasing GAE/Bingo, we received a number of worried correspondences from various very worried correspondents. It seems that GAE/Bingo, along with practically every other A/B testing framework out there, violates some purist principles of how to do significance testing.
The crux of the argument, reworded so simply that I’m pretty sure all statisticians (I admittedly know nothing...
October 2011
1 post
App Engine Performance Hacks You'll Probably Never...
Building GAE/Bingo required reaching into the bag of performance tricks a couple times. We needed long-term persistence of the data behind many A/B experiments, with stats accumulating at 500 reqs/sec, without slowing down pageloads.
Some of the wabbits we pulled out of the hat are pretty cool. Some are probably really stupid but get the job done. I’ll throw a couple your way and let you...
September 2011
1 post
A/Bingo split testing now on App Engine, built for...
Continuing my trend of straight-up copying the work of the smartest people I know, I recently decided to tackle Khan Academy’s A/B testing problem (we didn’t have any A/B testing) by bringing Patrick McKenzie’s A/Bingo into App Engine land.
So here you go: GAE/Bingo is released and should get anyone on App Engine up and A/B testing in minutes. It’s currently in production...
August 2011
1 post
Khan Academy Internship, Summer '11
Want to be handed a major portion of Khan Academy ownership, ridiculously high expectations, and a bunch of mentorship from our full-time devs? Sign up now. We believe anybody can help the world get a great education, and we accept interns year-round.
I can’t remember a time in my history of small company software development that hasn’t felt like sitting in a rickety donkey kong...
July 2011
2 posts
Fast and slow queues on App Engine
tl;dr — If you’re using task queues on App Engine and your task execution speeds vary greatly, you can get yourself into serious performance trouble. We addressed this by explicitly separating fast and slow tasks, and we released a little utility to help you do the same.
Only one user has ever earned this badge
Quick story: even though we put in a lot of work making sure Khan...
App Engine Mini Profiler and Temporary Redirects
We just made a small change to Google App Engine Mini Profiler to deal with the fact that spitting out profiling stats on every rendered page still misses all the pages that aren’t rendered: the POST side of temporary redirects.
It’s super common to submit a POST and redirect to a GET. When using a profiler that spits out performance data about the currently rendered page, you lose...
June 2011
3 posts
Google App Engine Mini Profiler
Patrick Bateman: (Looking at the business card) “Look at that subtle off-white coloring. The tasteful thickness of it. Oh my god…”
I finally understood the full extent of Bateman’s jealousy recently when reading about the MVC Mini Profiler created by the Stack Exchange team. My eyes scanned all the features, and I started to lose my mind:
“…live, instant...
Connecting the students
We’re thinking about how to connect our community of students, and there’s a lot we don’t know. Here’s the little we do:
Students have watched Sal’s videos over 60,000,000 times and have answered 75,000,000 practice problems.
This is a massive amount of learning, but much of it has been done by students who are relatively disconnected from their peers.
Nothing...
Opening up the Khan Academy API
As soon as I joined KA, I began receiving emails from people asking to integrate with Sal’s content in various ways: in mobile apps, in search engine results, in other educational sites. So we threw together the quickest, ugliest API we could get away with at the time in order to give other developers access to our playlist and video library.
It’s now a few months later, and that...
May 2011
3 posts
Pickles and memcache
My recent time spent creating a hack that I’d later delete taught me a bunch about Python and Django. One cool little trick I wound up using was a dictionary that automatically pickles and unpickles its contents.
I needed memcache to quickly deserialize a dictionary of values, and it wasn’t always guaranteed that I would use all of the objects in the dictionary with each request....
How much I enjoy deleting my own code
I’m not sure why a few months makes such a difference in how I feel about removing my own work. Maybe there’s some intuition that code hasn’t gotten to serve its real purpose without being out in the wild for a couple months. I don’t know. I do know that after a few months, when I’m free from the worry that I’ve recently wasted my time, nothing feels better than...
Raising expectations for educational tech speed
Khan Academy and the rising ecosystem of edutech startups aren’t going to come out of the gate with any perfect educational solutions (or anything close). We can raise the bar by building teams and products that move quickly in response to student data and the needs of learners. And teachers can, eventually, start to expect this well-deserved treatment instead of reeling in shock when their...
April 2011
4 posts
Profiling client-side performance over time
Steve Souders leads the way when it comes to client-side performance. A few years ago, his High Performance Web Sites book drilled the following into my head:
Most devs spend time optimizing server-side performance, but your users probably spend way more time waiting around for all types of client-side junk.
That book paved the way for great client-side perf tools like YSlow and, not long...
JS/CSS packaging to minimize requests and randomly...
From Joel’s recent IAmA on reddit, w/ ellipses thrown in haphazardly:
“What kind of sacrifices do you make on stackoverflow.com in order to obtain speed/efficiency?”
“Heavy caching…lavish spending…a willingness to let developers spend time on optimization.”
The performance benefits of packaging up your bunches of javascript and css files into as few...
More caching in Google App Engine
(This is a follow-up to Layer Caching in App Engine with memcache and Cachepy.)
The previous post on layer_cache was all about making it quick’n’easy to cache a function’s result across multiple requests using both memcache and App Engine instance memory:
@layer_cache.cache()
def gather_monkeys():
...gather...
As everybody who’s typed enough caching Hail Marys...
Keeping khanacademy.org synced with Sal
I just stumbled across Sal’s latest video on glucose, insulin, and diabetes, thought it was cool, and checked the Khan Academy homepage to make sure it was listed. It wasn’t. So I hit our magic khanacademy.org/sync-yoself-with-the-youtubez URL and in a couple minutes all was good. This wasn’t always always possible.
Those who keep up with his video production know that Sal...
March 2011
4 posts
Code Reviews as Relationship Builders - A Few Tips
I have code reviews on the brain as we build the Khan Academy team and prepare for our first class of summer interns. Code reviews are an important everyday tool for most tech teams, and I think they’re pretty much indispensable when absorbing the talents of a new developer.
Unfortunately, code reviews can also go very wrong. Taking a close, editorial look at the product of a...
Khan Academy's code now lives in Mercurial via...
Yesterday, we switched* our Subversion repository over to Mercurial and hosted it on Kiln. Check ‘er out: http://khanacademy.kilnhg.com
We switched for a number of reasons:
DVCS is better. If you disagree, you might be on the wrong blog. Here.
We’re a geographically separate team at the moment, and the more powerful our code review tools, the better.
We have a number of different...
Should Khan Academy follow Stack Overflow's...
Stack Overflow has a slick way of encouraging users to answer questions: you don’t need to sign up to start interacting with their
sites in meaningful, persistent ways. You can start building
reputation and earning badges without ever registering an account.
Then, when you want to make your (suddenly interesting) identity more
permanent, you can easily register and keep all your previous...
Khan Academy and LASD Pilot on Gates Notes →
khanacademy:
A few weeks ago, the Gates Notes team came down and interviewed the team here at the Khan Academy and all of the great students, teachers, and administrators participating in the pilot program over at the LASD. We’re excited to be able to share this with everyone because of how this experience has helped to inform many of the recent improvements to the Khan Academy. The Gates...
February 2011
5 posts
Using the jQuery Waypoints plugin on...
A few days ago a lightweight jQuery plugin — Waypoints — was launched via an announcement on Hacker News. Waypoints lets you run an arbitrary javascript function whenever you scroll to a particular element on the page.
$('#monkeys').waypoint(function() { console.log('You scrolled to the monkeys.'); });
At the time, Jason and I had been discussing an idea for the redesigned Khan...
Shipping is one heck of a drug
“Feels good to ship. It’s a drug to me.”
Jason sent me that text this morning after shipping the new khanacademy.org design last night. It rang true. After embracing the practice of continuous deployment so much that we sometimes ship new versions of Khan Academy 7 or 8 times a day, a 24 hour time span can start to feel dark and depressing if we haven’t made our...
In any language you want: Khan Academy interviews
I’ve been doing interview after interview for Khan Academy devs and Summer ‘11 interns. I come from the Joel school of interviewing, so I spend most of this time writing code side-by-side with our candidates.
I’m letting candidates use the language of their choice to solve our challenges and want to share some results.
At Fog Creek, candidates are warned that interview...
So much of what we're doing is obvious
I’ve heard Sal exclaim a couple times that everything we’re doing at the Khan Academy is obvious. Everyone involved agrees. Instead of struggling to invent something brand new, it feels like we’re coloring in an outline that should’ve never been empty in the first place.
If you sat around the table with a bunch of hacker geek types and said “teachers’ software...
Great lengths for immediate feedback: reporting on...
Can you imagine unlocking a new achievement or shooting the hat off your enemy in a Red Dead Redemption duel and not getting immediate feedback? The game would straight-up be considered broken, no questions asked.
Yet, plenty of game-like mechanics systems on the web are not immediate. Most badge systems make you wait a while before being awarded with the achievement you’ve just...
January 2011
8 posts
Messy solutions work just as well
I absolutely loved this bit from Marco Arment’s recent interview w/ Rands:
The [Instapaper] bookmarklet has a mechanism to save pages from sites that require logins…
The way it does this is ridiculous: instead of calling a simple GET request to save the page, since an entire page’s contents would quickly overrun any URL-length limits in the stack, it injects a FORM with a POST...
To all the big boys solving the internet's...
Solve time zones too.
Khan Academy lets users login using either their Google or Facebook account. Only one of those identity providers gives us access to the current user’s time zone offset. It’s not Google.
This drives me nuts because dealing with time zones is a ridiculous developer time sink, there’s no reason to ask our users to give us their TZ info again when...
Students Need Better Self-Assessment Tools. Khan...
“You make what you measure” is such a popular belief around the tech community that it’s become something of a status symbol to design and blog about your custom, gorgeous, flat-screen status board that sits in the hallway and shows off your company’s key metrics. The belief is that just by measuring and drawing daily attention to your most important numbers, they’ll...
Congratulations to the Newest Member of an Elite...
My congrats goes out to a hard-working Khan Academy user (and frequenter of this blog) who recently joined a very elite (5-person) group by grabbing the Millionaire badge.
No easy task.
Great White Sharks are the Best Side Project...
You know those absolutely insane video clips of great white sharks breaching into the air while eating poor defenseless seals? The cameras used to film them were capturing so many frames per second that they couldn’t keep all the data on the boat or stream it anywhere else fast enough, so they constantly buffered the past few seconds and dumped the rest of the footage.
Food Chain: Great...
Teachers have to be fearless
…to dive into a new way of doing things as radical as the Khan Academy. To integrate Khan Academy usage deep into the core of existing class structure requires more than just mental agility. It requires a willingness to experiment and an excitement about being part of something new and big in education.
That’s exactly what’s happening in our Los Altos School District pilot, and...
Calling all Masters of Algebra
Who’s gonna be first? The summative challenges Jason just blogged about have been live for about 10 hours. Arithmetic, Pre-algebra, and Trig have all been conquered by at least a few brave learners.
How long will it last?
The 150-problem Algebra Challenge stands strong (update) was awarded to the first user ~20 hours after the challenge launched.
Badges on App Engine: Implementing a real-time...
Building a badge system that gives feedback in real-time on top of Google App Engine involved more than one technical challenge worth sharing. This long post winds in and out of geeky tricks we used for the Khan Academy badges, ranging from protocol buffer serialization for fast memcache communication to App Engine’s mapreduce framework. Everything mentioned below is qualified with the...
December 2010
7 posts
Badge feedback from Facebook, Twitter, and Google...
Round 1 seems pretty successful with the early adopters, but they’re the easy crowd.
Meanwhile, Google Analytics has made it pretty obvious that even those users who are interested and technically savvy don’t necessarily completely understand what’s going on, and we’ll need to offer a much better explanation of the badging system soon.
These keyword hits have been...
Good Behavior Bad Behavior
We knew when we launched badges for Khan Academy that it would be tough (impossible) to design an incentive system without encouraging some bad behavior.
Due to some direct or side effect of new incentives in the system, users will become preoccupied with reaping rewards that aren’t aligned with what we consider most important.
Unwanted “Grinding” Behavior
When working through...
Khan Academy now has badges
…and students are in the hunt. It’s been a few hours since we launched badges for Khan Academy, and we’re already seeing some interesting behavior from reward chasers.
We stuck with our current astronomical theme because it’s kinda cool and why not?
We launched with all sorts of badges that represent our best first guess of behavior we want to encourage. We know...
Cannot multitask.
If I ever have to start and stop some long-running build or task, my productivity is destroyed in an unreasonable way.
Right now we run ~weekly updates that sync up the Khan Academy datastore’s list of videos with new YouTube content (which Sal uploads at a relentlessly breakneck pace). The whole process involves a couple quick manual steps and up to 30 minutes of waiting. If I were any...
Layer Caching in App Engine with memcache and...
The Khan Academy app is in need of a solid caching solution so it can grow new features without paying out performance losses.
All the best performing web apps in the world cache out the wazoo.
Whether they’re straight serving up static HTML files, caching everything possible into application memory, or shoving terabyte after terabyte into shared memcached, the top dev teams in the world...
Sprint sprint sprint, a hint of what's to come
The Khan Academy team has been sprinting like crazy to learn as much as possible from our tight iteration loop with our pilot school system.
…water…
It’s a blast. We’ve either launched or are about to launch:
Improved charting/reporting for teachers. It’s hard to talk about all the data reporting features we have implemented without wanting to immediately concede...