I was playing around with Merb some last night but I was running into a problem when I tried to utilize my models in a Rake task. Searching around I finally came across the Merb version of the task foo => :environment, it’s task foo => :merb_env. Now things are going much more smoothly.
One issue when you have existing user base is how to roll out new features with minimal impact on servers and user experience. When Facebook rolled out Facebook Chat they used a pretty good system of figuring out how things can scale.
The secret for going from zero to seventy million users overnight is to avoid doing it all in one fell swoop. We chose to simulate the impact of many real users hitting many machines by means of a “dark launch” period in which Facebook pages would make connections to the chat servers, query for presence information and simulate message sends without a single UI element drawn on the page.
At EarthLink, I helped build a new user preference system, replacing an older system. Obviously, you have to switch the systems without a hiccup so no one would notice. The way we did was very similar to Facebook’s. Each time a preference was saved in the old system, we also saved it in the new system, allowing us to see the impact of the new system on the user experience. It helped us shake out bugs which wouldn’t have shown themselves during basic load testing. Lauching a new app is easy since you can scale it up but it makes things much more interesting when you have to deal with existing users.
Kevin Burton has posted his slides from the MySQL conference. It’s based on his blog aggregator, Spinn3r which uses MySQL in write-heavy processes instead of the usual read path.
Since everyone else is doing it:
a21772:~ jlucas$ history|awk '{a[$2]++} END{for(i in a){printf "%5d\t%s \n",a[i],i}}'|sort -rn|head
126 rake
121 svn
69 vi
57 ./script/console
21 ./script/server
17 cd
15 ssh
13 cap
10 ls
7 mysql
As you can see, most of my work lately has been Ruby / Rails. No complaints from me.
Everyone talked about the App Engine yesterday and folks seem to go in either of the extremes of loving it or hating it. I’ve seen conspiracy theories about giving Google access to your data but also how this could be the start of many simple applications that wouldn’t see the light of day without something like this.
And there is the response from Giles. I work with Giles, respect him a ton but I think he’s wrong here or at least it’s too early to see if he is right or not.
Google App Engine is a joke. You can’t create apps in Rails, Haskell, Erlang, Seaside, Perl, PHP, or Lisp. You can’t even leverage Unix. You get all the restrictive corporate BS you’d have to deal with if you worked at Google, but you don’t get the stock options, the salary, the free food, or even a parking space for your trouble. Packing Google with partisans who favored a particular language to unreasonable degrees resulted in a product which just isn’t even worth considering. This is why polyglot programming matters. The age of one dominant language is over, and Google’s attempt to make Python numero uno just means wasted time and wasted money.
I think the focus on Python-only is missing a couple of things that seem fairly obviously. One, the main config app.yaml file has an option for saying which runtime is being used and states that other languages / runtimes may be supported in the future. And yes, I do see that it isn’t for sure but come on, do you really think Google will give that kind of additional momentum to Amazon or any other competitors. The second thing is that most developers know that getting something out the door is more important that trying to get all possible features included before launch. Why should this be considered any different?
We know that Google has the Python expertise, why is it shocking that Python is the first language supported? Why would they want to offer a Rails environment and get slammed for it not being totally ready. Why not get the bugs shaken out with now before opening it up to other languages?
Of course, I could totally be wrong and this is an end-game play by Google but it seems way too early for that.
Looks like an action-packed couple of days at the end of June up in the Bay. First, there’s the O’Reilly Velocity conference and then on the 25th is Structure ’08 conference. I’m not sure if I can go to either of them but they should be interesting.
So no, we don’t hire architects. We hire developers. In a small team, there is no room for management deadwood. Everybody pays their own freight. The more senior you are, the more you get to help and coach and mentor others. Leading means enabling others to do their job and make good decisions on their own, not cutting their food and handing off work.
I think I’m going to paste this blurb up outside my office. There’s this undercurrent going around the office about the need for an architect who would help guide us through the next phases of development. Thankfully, this hasn’t been done yet and I’ll fight it as long as I can.
Blaine Cook from Twitter released Starling last week. He describes it as:
Starling is a light-weight persistent queue server that speaks the MemCache protocol. It was built to drive Twitter’s backend, and is in production across Twitter’s cluster.
I’m always a sucker for new infrastructure code but the problem is trying to figure out how to see if it will fit within whatever current architecture is already in place. I think though I might have found somewhere.
We have an aggregator grabbing various RSS feeds both internally and externally. Right now, during the parsing / adding to the database, we send off the information to our indexer for later searching. This is basically how the Topics pages are built dynamically.
Instead of sending the data during run-time, maybe it would be better to just add it to a queue with Starling for later processing. A job in the indexer could check the queue and index anything that’s there. This also would allow other applications to add things ready for indexing.
We launched something officially this week, http://topics.latimes.com. It actually went out mid-December but it isn’t official until it was linked to from the main Entertainment section, which was given a facelift in case you didn’t notice.