May 2013
1 post
You must stop coding, you must keep coding
Our dev team’s grown to twenty-three and when the summer intern zenith hits we’ll be — this is weird to type — forty for a few frightening days.
My biggest personal challenge now, by far: How do you stay connected to a product amidst all this scary m͓͇̘̗a̴̹̠̜̝͖̘n̩͙̘͙̫ͅa̬͙̖̬͍̦͔g͓̼͈͇̙i̪̺̘̙̻͘n͕̠̰̦̗g̗? Do you launch the damn editor? Write a single line of code?
I’ve been trying...
March 2013
2 posts
The Absolute Minimum Every Khan Academy Developer...
Heard a bit of an interview with Dick Costolo while driving home. He was telling the story of why he started his “Managing at Twitter” course that he personally teaches to all new managers. It went something like so:
An engineer came up to me who had recently switched from one Twitter dev team to another. He said, “My old manager used to hold 1:1 meetings with me. My new one...
Breaking down Amazon's mega dropdown
The hover effects on Amazon’s big ‘ole “Shop by Department” mega dropdown are super fast. Look’it how quick each submenu fills in as your mouse moves down the list:
It’s instant. I got nerd sniped by this. Most dropdown menus have to include a bit of a delay when activating submenus. Here’s an old Khan Academy dropdown as an example:
See the...
February 2013
2 posts
Meritocracy and Discrimination in Tech
I’ve been the jerk who denies a candidate their hopes for an internship hundreds of times. That’s not a point of pride – just a consequence of helping build eight (nine?) classes of interns at two small companies that both happen to receive up to 500 intern applications every month.
I’ve obsessed over finding ways to build internship classes full of the best developers in an...
Hacking at a nonprofit is just like its...
Paul Graham claims nonprofit startups are similar to their counterparts. As soon as I read the headline I realized I’ve been answering a different version of this question in every recruiting conversation I’ve had for the past two years:
“What’s it like to be a hacker at a nonprofit?”
I’ll answer candidly based on my stories on each side of the profit divide....
January 2013
2 posts
Pending queues and loading requests on App Engine
Sometimes it feels like we’re playing a rousing of guess’n’check when adjusting our App Engine performance settings. The request scheduler is a bit of a black box, so it can be hard to know which knobs we should twist and how far.
But there are two really straightforward signs any App Engine dev should watch out for. Both are a big red flag signaling a configuration not...
A sampling profiler for App Engine
I’d been getting a bit frustrated trying to figure out where time was being spent during a few of Khan Academy’s longer running requests. App Engine Mini Profiler usually works great, but these specific requests seemed to be spending a lot of time serializing objects into JSON representations. JSON serialization, by its nature, involves tons and tons of nested function calls. Anathema...
December 2012
1 post
"I don't understand"
I’ve noticed that the most mature and accomplished developers I’ve worked with are also those who most frequently say “I don’t understand” when they’re listening to a technical explanation. This has been the case with coworkers both at Fog Creek and at Khan Academy.
In one way, it’s counterintuitive. Shouldn’t the senior devs already know...
November 2012
3 posts
Dangers of using memcache counters for a/b tests
We ran into a hairy problem with our App Engine A/B testing framework a few months ago. It’s a problem unique to those relying on memcache to increment counters with statistical meaning in a shared resource environment like App Engine.
Since there are more live-action teenage mutant ninja turtle films than people who fit that bill (sorry, one person on the internet, I know you matter),...
Khan Academy Internship, Summer '12
Teaser: we’re full steam ahead hiring for the summer ‘13 class of interns. Apply.
We had fifteen interns this summer and around that many full-timers.
It’d be silly for me to try to write a summary of what was accomplished. That task was doable last summer and fall, but this time too much was built. If I made a list of it all, I’d ramble even worse than normal.
...
A maker-manager's schedule
Paul Graham’s Maker’s Schedule, Manager’s Schedule meant a lot to me. It meant a lot to anybody who’s ever been inexplicably frustrated by a kind-hearted coworker interrupting them, or felt like an ass for turning down a request to grab a 20 minute coffee with an acquiantance.
It meant a lot because it explained the frustration. It justified (as much as can be justified)...
September 2012
1 post
What traffic from 60 Minutes looks like
Yesterday evening Khan Academy was featured on 60 Minutes. It was a rerun.
Ben Eater recorded our real-time active visitors during the East Coast feed — here it is sped up 10x.
Last time In March, when the clip first aired, I had Camtasia open and was recording my view of Google Analytics’ real-time visitor data. I was excited to share a view into the traffic Sanjay Gupta commands...
July 2012
3 posts
Lessons learned A/B testing with GAE/Bingo
Almost a year ago we released a version of Patrick McKenzie’s A/Bingo, built for Khan Academy and others in need of an App Engine-optimized split testing tool. It’s been in constant production use at Khan since then. We’ve made mistakes, learned lessons, and spent plenty of time scratching our heads pondering results.
Thanks to those lessons, the current version of GAE/Bingo...
How we ran the first Khan Academy Healthy...
This weekend was our first hackathon. Despite not knowing WTF we were doing as organizers, I’m calling it a huge success. The creations demo’d yesterday blew me away (more to come on that).
When we started trying to puzzle out the few details that need to be puzzled out when running a tiny, private hackathon, we stole plenty from the shared experiences of others. Figured I’d...
My Compute Engine Pipe Dream
I’m gonna don my rose-colored glasses and take a look at Google’s newly announced Compute Engine. As the majority of Khan Academy runs on App Engine, I’m very excited about a specific possibility here.
While I’m aware of the fact that Compute Engine’s list of use cases is currently aimed at backend data analysis type stuff, I’m gonna take a guess and say...
May 2012
2 posts
App Engine Performance Essentials You'll Certainly...
Update: the bad fetch performance described in this post has been fixed by the ever responsive App Engine team (see comments below). However, this read is still useful and I still recommend using .run instead of .fetch to stay on top of current App Engine best practices.
It’s kinda cheating to call this Part II of App Engine Performace Hacks You’ll Probably Never Need, but I recently...
How intern mentorship works at Khan Academy
Three of our twelve(‽) summer interns have arrived. It’s high time I share how we mentor interns before I’m spending all my time swatting sharks and helicopters out of the air. These tips also apply to new full-time hires, of course, but for now I’ve got interns on the brain.
1. Make the mentorship relationship explicit
At some point in a new employee’s first day they...
April 2012
1 post
"Step one? FIX. Step two? IT."
“Step three? FIXIT. Now repeat till it is FIXED.”
Wise words from Kenan, patron saint of the first Khan Academy fixit.
Fixit day is just one more in the long list of solid dev lessons I’ve been learning from the Googlers around here. Since I couldn’t find a reputable source explaining the fixit culture (who reads the NY Times?), I figured it’s my duty to share.
...
March 2012
1 post
Teacher effectiveness ratings, programmers, and...
Today I was doing whatever it is I do when I ran across this link from Joel:
NYC teacher “effectiveness” ratings are bogus, and the data prove it garyrubinstein.teachforus.org/2012/02/28/ana…— Joel Spolsky (@spolsky) March 1, 2012
…and my brain started pattern matching. Replace “NYC teacher” with “programmer” in this tweet, and you’d be in...
February 2012
1 post
Required code reviews
This is the story of the growing Khan Academy team converting me into a passionate fan of requiring a code review for every single changeset.
Those who have worked with me know that it’s a surprising position for me to take. On the spectrum of “Follows good development practices even if it slows down the product” to “Just ship the thing, code doesn’t matter, only...
January 2012
2 posts
Sharing the inspiring, personal stories of Khan...
We have these emails hanging up all over our office, sent in from Khan Academy users with incredible, personal stories to tell. Every time I read a new one I’m emotionally affected, which means my robot emotion chip is faulty.
So when some curious soul (like a reporter) wanders in and asks me, “How will you know if Khan Academy is really successful?” I always answer their...
Khan Academy Internship, Fall '11
I already can’t wait to drop some major challenges in the laps of our two incoming Fall interns to see what they can build.
- Khan Academy Internship, Summer ‘11
Check!
David Hu and Julian Pulgarin stepped up to the plate this Fall during their coopsinternships from University of Waterloo. We call ‘em internships because we’re from Amurrrrica, you crazy Canadians.
When...
December 2011
3 posts
.end() makes jQuery DOM traversal beautiful
This is my new favorite jQuery trick. I just learned it this year and have mentioned it in enough code reviews to decide it’s worth sharing.
When manipulating the DOM with jQuery, you often see code that looks something like:
$("#container").show();
$("#container .error").hide();
$("#container .zoo").css("background-color", "white");
$("#container .zoo...
Laughing at Others' Code
Those who have worked with me will know that I’m an expert on this topic because my code gets laughed at all the time.
I’ve seen Good laughing and Bad laughing. Good is what I imagine happens when Robert De Niro sits next to Al Pacino as they’re watching Al’s cameo in some trainwreck of an Adam Sandler movie, and Bob turns to Al smiling and says, “This is...
After giving logged out users access to pretty...
We had some heated debates a while ago about what would happen if we opened up all of Khan Academy’s content for logged out users. Sal’s videos have always been open in this way, of course, but all the interactive exercises and statistical tracking and badges and stuff required an account.
It felt like the right move when we reconfirmed our belief that educational content should be...
November 2011
2 posts
How to make "consecutive days of Khan Academy...
Think how cool it would be if users were rewarded after learning for X consecutive days.
Think about implementation for a second and then go, “Sweet! This’ll be trivial Let’s knock it out Here we go I’ve got things to do!”
Write a line of code or two.
Get sucker-punched by the gruesome specter of timezones when realizing that a student doing work at 8am on...
A/B testing still works. [Sarcastic *PHEW*].
After releasing GAE/Bingo, we received a number of worried correspondences from various very worried correspondents. It seems that GAE/Bingo, along with practically every other A/B testing framework out there, violates some purist principles of how to do significance testing.
The crux of the argument, reworded so simply that I’m pretty sure all statisticians (I admittedly know nothing...
October 2011
1 post
App Engine Performance Hacks You'll Probably Never...
Building GAE/Bingo required reaching into the bag of performance tricks a couple times. We needed long-term persistence of the data behind many A/B experiments, with stats accumulating at 500 reqs/sec, without slowing down pageloads.
Some of the wabbits we pulled out of the hat are pretty cool. Some are probably really stupid but get the job done. I’ll throw a couple your way and let you...
September 2011
1 post
A/Bingo split testing now on App Engine, built for...
Continuing my trend of straight-up copying the work of the smartest people I know, I recently decided to tackle Khan Academy’s A/B testing problem (we didn’t have any A/B testing) by bringing Patrick McKenzie’s A/Bingo into App Engine land.
So here you go: GAE/Bingo is released and should get anyone on App Engine up and A/B testing in minutes. It’s currently in production...
August 2011
1 post
Khan Academy Internship, Summer '11
Want to be handed a major portion of Khan Academy ownership, ridiculously high expectations, and a bunch of mentorship from our full-time devs? Sign up now. We believe anybody can help the world get a great education, and we accept interns year-round.
I can’t remember a time in my history of small company software development that hasn’t felt like sitting in a rickety donkey kong...
July 2011
2 posts
Fast and slow queues on App Engine
tl;dr — If you’re using task queues on App Engine and your task execution speeds vary greatly, you can get yourself into serious performance trouble. We addressed this by explicitly separating fast and slow tasks, and we released a little utility to help you do the same.
Only one user has ever earned this badge
Quick story: even though we put in a lot of work making sure Khan...
App Engine Mini Profiler and Temporary Redirects
We just made a small change to Google App Engine Mini Profiler to deal with the fact that spitting out profiling stats on every rendered page still misses all the pages that aren’t rendered: the POST side of temporary redirects.
It’s super common to submit a POST and redirect to a GET. When using a profiler that spits out performance data about the currently rendered page, you lose...
June 2011
3 posts
Google App Engine Mini Profiler
Patrick Bateman: (Looking at the business card) “Look at that subtle off-white coloring. The tasteful thickness of it. Oh my god…”
I finally understood the full extent of Bateman’s jealousy recently when reading about the MVC Mini Profiler created by the Stack Exchange team. My eyes scanned all the features, and I started to lose my mind:
“…live, instant...
Connecting the students
We’re thinking about how to connect our community of students, and there’s a lot we don’t know. Here’s the little we do:
Students have watched Sal’s videos over 60,000,000 times and have answered 75,000,000 practice problems.
This is a massive amount of learning, but much of it has been done by students who are relatively disconnected from their peers.
Nothing...
Opening up the Khan Academy API
As soon as I joined KA, I began receiving emails from people asking to integrate with Sal’s content in various ways: in mobile apps, in search engine results, in other educational sites. So we threw together the quickest, ugliest API we could get away with at the time in order to give other developers access to our playlist and video library.
It’s now a few months later, and that...
May 2011
3 posts
Pickles and memcache
My recent time spent creating a hack that I’d later delete taught me a bunch about Python and Django. One cool little trick I wound up using was a dictionary that automatically pickles and unpickles its contents.
I needed memcache to quickly deserialize a dictionary of values, and it wasn’t always guaranteed that I would use all of the objects in the dictionary with each request....
How much I enjoy deleting my own code
I’m not sure why a few months makes such a difference in how I feel about removing my own work. Maybe there’s some intuition that code hasn’t gotten to serve its real purpose without being out in the wild for a couple months. I don’t know. I do know that after a few months, when I’m free from the worry that I’ve recently wasted my time, nothing feels better than...
Raising expectations for educational tech speed
Khan Academy and the rising ecosystem of edutech startups aren’t going to come out of the gate with any perfect educational solutions (or anything close). We can raise the bar by building teams and products that move quickly in response to student data and the needs of learners. And teachers can, eventually, start to expect this well-deserved treatment instead of reeling in shock when their...
April 2011
4 posts
Profiling client-side performance over time
Steve Souders leads the way when it comes to client-side performance. A few years ago, his High Performance Web Sites book drilled the following into my head:
Most devs spend time optimizing server-side performance, but your users probably spend way more time waiting around for all types of client-side junk.
That book paved the way for great client-side perf tools like YSlow and, not long...
JS/CSS packaging to minimize requests and randomly...
From Joel’s recent IAmA on reddit, w/ ellipses thrown in haphazardly:
“What kind of sacrifices do you make on stackoverflow.com in order to obtain speed/efficiency?”
“Heavy caching…lavish spending…a willingness to let developers spend time on optimization.”
The performance benefits of packaging up your bunches of javascript and css files into as few...
More caching in Google App Engine
(This is a follow-up to Layer Caching in App Engine with memcache and Cachepy.)
The previous post on layer_cache was all about making it quick’n’easy to cache a function’s result across multiple requests using both memcache and App Engine instance memory:
@layer_cache.cache()
def gather_monkeys():
...gather...
As everybody who’s typed enough caching Hail Marys...
Keeping khanacademy.org synced with Sal
I just stumbled across Sal’s latest video on glucose, insulin, and diabetes, thought it was cool, and checked the Khan Academy homepage to make sure it was listed. It wasn’t. So I hit our magic khanacademy.org/sync-yoself-with-the-youtubez URL and in a couple minutes all was good. This wasn’t always always possible.
Those who keep up with his video production know that Sal...
March 2011
4 posts
Code Reviews as Relationship Builders - A Few Tips
I have code reviews on the brain as we build the Khan Academy team and prepare for our first class of summer interns. Code reviews are an important everyday tool for most tech teams, and I think they’re pretty much indispensable when absorbing the talents of a new developer.
Unfortunately, code reviews can also go very wrong. Taking a close, editorial look at the product of a...
Khan Academy's code now lives in Mercurial via...
Yesterday, we switched* our Subversion repository over to Mercurial and hosted it on Kiln. Check ‘er out: http://khanacademy.kilnhg.com
We switched for a number of reasons:
DVCS is better. If you disagree, you might be on the wrong blog. Here.
We’re a geographically separate team at the moment, and the more powerful our code review tools, the better.
We have a number of different...
Should Khan Academy follow Stack Overflow's...
Stack Overflow has a slick way of encouraging users to answer questions: you don’t need to sign up to start interacting with their
sites in meaningful, persistent ways. You can start building
reputation and earning badges without ever registering an account.
Then, when you want to make your (suddenly interesting) identity more
permanent, you can easily register and keep all your previous...
Khan Academy and LASD Pilot on Gates Notes →
khanacademy:
A few weeks ago, the Gates Notes team came down and interviewed the team here at the Khan Academy and all of the great students, teachers, and administrators participating in the pilot program over at the LASD. We’re excited to be able to share this with everyone because of how this experience has helped to inform many of the recent improvements to the Khan Academy. The Gates...
February 2011
5 posts
Using the jQuery Waypoints plugin on...
A few days ago a lightweight jQuery plugin — Waypoints — was launched via an announcement on Hacker News. Waypoints lets you run an arbitrary javascript function whenever you scroll to a particular element on the page.
$('#monkeys').waypoint(function() { console.log('You scrolled to the monkeys.'); });
At the time, Jason and I had been discussing an idea for the redesigned Khan...
Shipping is one heck of a drug
“Feels good to ship. It’s a drug to me.”
Jason sent me that text this morning after shipping the new khanacademy.org design last night. It rang true. After embracing the practice of continuous deployment so much that we sometimes ship new versions of Khan Academy 7 or 8 times a day, a 24 hour time span can start to feel dark and depressing if we haven’t made our...
In any language you want: Khan Academy interviews
I’ve been doing interview after interview for Khan Academy devs and Summer ‘11 interns. I come from the Joel school of interviewing, so I spend most of this time writing code side-by-side with our candidates.
I’m letting candidates use the language of their choice to solve our challenges and want to share some results.
At Fog Creek, candidates are warned that interview...
So much of what we're doing is obvious
I’ve heard Sal exclaim a couple times that everything we’re doing at the Khan Academy is obvious. Everyone involved agrees. Instead of struggling to invent something brand new, it feels like we’re coloring in an outline that should’ve never been empty in the first place.
If you sat around the table with a bunch of hacker geek types and said “teachers’ software...
