They’re really expensive. They burn your most precious resources when running a startup: good will, and time. Your support staff has to address the issues (while people are yelling at them), your engineers are in the database mucking about with columns, until they finally break down about build an unbanning tool which inevitably doesn’t scale to really massive attacks, or new interesting attack vectors, which means you’re either back monkeying with the live databases or you’ve now got a team of engineers dedicated just to building tools to remediate false positives. And now you’re burning engineer cycles, engineering motivation (cleaning up mistakes sucks), staff satisfaction AND community good will. That’s the definition of expensive.

Unfortunately, there’s just no good answer. You can’t ignore jerks forever and you can’t try to outmaneuver them too early. The answer isn’t purely product design, community management, or tools. It’s not all about the monetary cost to your business, but it’s not all about the intangible cost.

Basically, it takes a really, really sharp product person to figure this out. I highly recommend them, if you have the means.

“Otis” is a song arranged and performed by two best pals, Jay-Z and Kanye West. It opens with a sample of Otis Redding (hence the title) singing “Try a little tenderness”. Opening with a sample like this tells us two things:

• Misters Z and West enjoy the music of Mr. Redding enough that they were compelled to include it in their own music.
• The gentlemen are also well connected and affluent, as not just everyone can afford to sample a legend like Redding in their music

A digression: sampling in hip-hop is one of its key characteristics and is of particular interest to nerds. It is a way that we can connect, through “nerding out” with the artist and find what it is that they respect and listen to. It is also a bit of a recursive structure; “Otis” samples Otis, Otis borrowed from gospel and blues, blues and gospel borrowed from traditional songs, etc. Finally, sampling is a recombinant form; in “Otis”, there is a verbatim sample in the opening bars, but the sample devolves to a looped-beat in the middle of the song and a mere sound-effect at the end of the song.

As Misters Z and West enter the song proper, the rappers trade verses about their affluence (“New watch alert, Hublot’s / Or the big face Rollie I got two of those”), the recursive (again, nerdy) deception they use to evade the papperazzi (“They ain’t see me ‘cuz I pulled up in my other Benz / Last week I was in my other other Benz”), a conflicting verse about how they would seek the paparazzi out (“Photo shoot fresh, looking like wealth / I’m ‘bout to call the paparazzi on myself”), more boasting of their affluence and skill (“Couture level flow, it’s never going on sale / Luxury rap, the Hermes of verses”), and such.

This song features a video, so we wouldn’t be properly doing a nerd dissection of it if we were to neglect that. It opens with our heroes approaching a Maybach sedan with a saw and a blow torch. Following a “car modification montage”, it appears the doors have been removed from the car and the front end of the car has been placed on the back, and vice versa. Another display of affluence, with perhaps a touch of hipster irony thrown in.

The video follows with various shots of our heroes rapping and driving the Maybach through an abandoned dock or airfield. Our heroes are in the front seat of the car and there are four models in the back seat, one seated precariously atop the one in the middle as our heroes make dangerous-looking manuvers in the car. At one point it appears they will lose a model through the door-less side of the car. At multiple points, it appears the boobs of the models might fight free of their loose fitting shirt. It should be noted that the appearance of a possible free boob could be considered quite progressive for a hip-hop music video.

The Maybach is, in my opinion, the most difficult to interpret signal the song and video send. Are we to understand that Misters Z and West are so affluent they can afford to put down six figures on the purchase and massively impractical modification of a high-end luxury car? Perhaps they had a spare one laying around and felt it would be a better use to destroy it than to leave it around. Or, perhaps this was a vehicle for a clever tax deduction?

Mr. West’s CPA: You’re going to owe a lot of tax on this purchase of your other-other Benz, ‘Ye.

Mr. West: What if I were to use it in a music video for the purposes of promoting my upcoming album?

Mr. West’s CPA: Well then you could depreciate it at 50% this year and 25% for the next two years, but you’re still going to owe a lot.

Mr. West: If I were to take it to a chop shop and have them put the ass-end of the car on the front and turn the doors into wings, could I depreciate it faster?

Mr. West’s CPA: throw some models in the back seat, and it *just might work*!

(Ed. as it turns out, the vehicle was to be auctioned and the proceeds donated to charity)

The other enigma of the video is the presence of comedian Aziz Ansari. Mr. Ansari has documented (Ed. hilariously) his friendship with Mr. West. Thus it is not shocking to see him appear in the video. He appears for only an instant, and his appearance marks the absence of the models in the rest of the video. Perhaps, we are to believe, Mr. Ansari is the pumpkin that the models turn into after some deadline has passed for Misters Z and West.

Despite, or perhaps because, of its mysteries, I find “Otis” is a fantastic piece of hip-hop production. The samples is well chosen and deconstructed, the verses are interesting (if mentally unchallenging), and the video is engaging to watch. I would easily rank it amongst the top songs of recent memory, were I one to make lists of top songs.

At 5am I jump out of bed and code for two hours, then get ready for work. I do this every day, including weekends.

I’ve done this at various times. It’s a fantastic hack, especially if you’re a morning person. No one’s around to bug you, nothing else is in your head. It’s just you, your project, and the earlybirds.

I highly recommend it if you have the means.

In the REST world, the resource is key, and it’s really tempting to simply look at a Django model and make a direct link between resources and models — one model, one resource. This fails, though, as soon as you need to provide any sort of aggregated resource, and it really fails with highly denormalized models. Think about a Superhero model: a single GET /heros/superman/ ought to return all his vital stats along with a list of related Power objects, a list of his related Friend objects, etc. So the data associated with a resource might actually come out of a bunch of models. Think select_related(), except arbitrary.

Mistaking the app’s internal model with what API users want to work with was the mistake I made on the first API I wrote.

Any big API is going to need to have dedicated servers that just serve API applications: the performance characteristics of large-scale APIs are so different from web apps in general that they almost always require separately-tuned servers.

This is how I prefer to roll my APIs lately. At the least, they should be a separate set of controllers. If you can extract a completely different application even better.

There’s a lot of experimentation in the wild attempting to figure out what works well here. You can bang out a bunch of presenter classes. You can describe and compose representations. You can go resource oriented.

I came across one yesterday that immediately caught my eye. You could just use `lambda` to implement a bunch of functions that present, decorate, or map objects from one representation to another. To borrow an example:

# Define a base representation
UrlsPresenter = lambda do
  {
    'self'    => "#{Gauges.api_url}/me",
    'gauges'  => "#{Gauges.api_url}/gauges",
    'clients' => "#{Gauges.api_url}/clients",
  }
end

# Compose the base representation with more data
UserPresenter = lambda do |user|
  {
    'id'          => user.id,
    'email'       => user.email,
    'name'        => user.name,
    'urls'        => UrlsPresenter.call
  }
end

# Pass an object to the presenter and convert it to JSON
UserPresenter[user].to_json

I love that this one adds no machinery and no state. Input, function, output. With just `lambda`, you can describe a bunch of transformations and string them together into meaningful and interesting pipelines. I'm experimenting with this now, hoping to find an interesting way that functional programming approaches can make it simpler to build APIs with Rails.

Dead-simple doubles

In my opinion, the most “wholly legitimate” frequent application of Class.new is in test code. It’s a great tool for creating test doubles, fakes, stubs, and mocks without the weight of pulling in a framework. To wit:

TinyFake = Class.new do

  def slow_operation
    "SO FAST"
  end

  def critical_operation
    @critical = true
  end

  def critical_called?
    @critical
  end

end

tiny_fake = TinyFake.new
tiny_fake.slow_operation
tiny_fake.critical_operation
tiny_fake.critical_called? == true

TinyFake functions as a fake and as a mock. We can call a dummy implementation of slow_operation without worrying about the snappiness of our suite. We can verify that a method was called in the verification section of our test method. Normally you would only do one of these things at a time, but this shows how easy it is to roll your own doubles, fakes, stubs, or mocks.

The thing I like about this approach over defining classes inside a test file or class is that it’s all scoped inside the method. We can assign the class to a local and keep the context for each test method small. This approach is also really great for testing mixins and parent classes; define a new class, add the desired functionality, and test to suit.

DSL internals

Rack and Resque are two examples of libraries that expose an API based largely on writing a class with a specific entry point. Rack middlewares are objects with a call method that generates a response based on an environment hash and any other middlewares that are contained within the middleware. Resque expects the classes that work through enqueued jobs define a perform method.

In practice, putting these methods in a class is the way to go. But, hypothetically, we are way too lazy to type class/end, or perhaps we want to wrap a bunch of standard instrumentation and logging around a simple chunk of code. In that case, we can write ourself a little shortcut:

module TinyDSL

  def self.performer(&block)
    c = Class.new
    c.class_eval { define_method(:perform, block) }
    c
  end

end

Thingy = TinyDSL.performer { |*args| p args }
Thingy.new.perform("one", 2, :three)

This little DSL gives us a shortcut for defining classes that implement whatever contract is expected of performer objects. From this humble beginning, we could mix in modules to add functionality around the performer, or we could pass a parent class to Class.new to make the generated class inherit from another class.

That leads us to the sort-of shortcoming of this particular application of Class.new: if the unique function of performer is to wrap a class around a method (for instance, as part of an API exported by another library), why not just subclass or mixin that functionality in the client application? This is the question you have to ask yourself when using Class.new in this way and decide if the metaprogramming is pulling its weight.

How Class.new is used in Sinatra

Sinatra is a little language for writing web applications. The language specifies how HTTP requests are mapped to blocks of Ruby. Originally, you wrote your Sinatra applications like so:

get '/'  { [200, {"Content-Type" => "text/plain"}, "Hello, world!"] }

Right before Sinatra 1.0, the team added a cleaner way to to build and compose applications as Ruby classes. It looks the same, except it happens inside the scope of a class instead of the global scope:

class SomeApp < Sinatra::Base

    get '/'  { [200, {"Content-Type" => "text/plain"}, "Hello, world!"] }

end

It turns out that the former is implemented in terms of the latter. When you use the old, global-level DSL, it creates a new class via Class.new(Sinatra::Base) and then class_evals a block into it to define the routes. Short, clever, effective: the best sort of Class.new.

So that’s how you might see Class.new used in the wild. As with any metaprogramming or construct labeled “Advanced (!)”, the main thing to keep in mind, when you use it or when you set upon refactoring an existing usage, is whether it is pulling its conceptual weight. If there’s a simpler way to use it, do that instead.

But sometimes a nail is, in fact, a nail.

Things I read

The most interesting fiction I read this year was Neuromancer. This one was probably more jaw dropping before The Matrix came out, but it was still interesting. That said, it reads like the Cliffs Notes version of Neal Stephenson.

The best non-fiction I read was Godël’s Proof. It’s a short, clear explanation of his approach to computability. Even if you’re medicore at math and proofs, like myself, this one will stick.

The best technical book I consumed this year was Smalltalk Best Practice Patterns. First off, I love Kent Beck’s concise but powerful writing style. Second off, this book is like discovering that someone wrote down a really good theory of the elements of software decades ago and no one told you about them. Third, you should get a copy of this book, look for the used ones.

Things I made this year

I piled a lot of the things I learned about infrastructure and shipping software at Gowalla into Mixing a Persistence Cocktail. How to think about scaling, how to ship incrementally, overcoming THE FEAR. It’s all there.

I large chunk of my time at work on Chronologic. I presented on it too. Then I open sourced it. We deployed it at Gowalla and it held up, but not without some rough spots. I presented on those too.

I did a lot of open source tinkering this year. A lot of it is half-baked, but at least it’s out there. That was a major personal goal for the year, so I’m glad I at least stuck my neck out there, even if I’m not rolling in kudos. Yet!

Things I wrote this year

Modulo a summer lull, I ended up doing a good bit of writing this year. The crowd favorites were Why metaprogram when you can program?, The Current and Future Ruby Platform
Cassandra at Gowalla, and Your Frienemy, the ORM. My personal favorites were The ear is connected to the brain, Post-hoc career advice for twenty-something Adam, and How to listen to Stravinsky’s Rite of Spring.

Of course, working at Gowalla this year was quite the ride. I wrote about that too, sometimes rather obliquely. Relentless shipping,
The pitfalls of growing a team, The guy doing the typing makes the call, Skip the hyperbole, Sleep is the best, and Don’t complain, make things better were all borne of things I learned over the course of the year.

If I had to pithily summarize the year, I’d tie it together under change. Change is good, challenging, frustrating, and inevitable. Better to change than not, though!

A picture can nullify a thousand talking points, no?

Long story short, I’ve been wanting to get back into music lately, but I want to learn something new. Something entirely mysterious to me. Given my recent fascination with hip-hop, I’m eager to try my hand at making the beats that form the musical basis of the form.

There are a lot of priors to cover (tinkering with various sequencers, drum machines, and synthesizers; steeping myself in sample culture; listening to the actual music and understanding its history), but I just made a short, mediocre little beat and put it on the internet. Herein, I reflect on making that little musical thing:

• I’m sure that, if I get serious about this, I’ll need real software like Ableton or Logic. But for my tinkering, it turns out GarageBand is sufficient. The included software instruments aren’t amazing or even idiomatic samples (no TR808, no “Apache” break included), but with a little bit of tinkering, they produce results.
• Laying a drum track down that is little more than a fancy click track helps to get started. GarageBand has a handy feature where you can define the a number of bars as a loop and then record multiple takes, review them, and discard the takes you don’t want.
• What an app lacks in samples you can make up in effects. Throwing a heavy dose of echo and a ridiculous helping of reverb made an otherwise pedestrian drum track way more interesting.
• I didn’t go into this with anything in my head that I wanted to make real. For the drum track, I ended up with a pretty typical beat. A little quantization made it end up sound better and more interesting than it really is. This process, manual input with some computer-assisted tweaking, produced way better results than the iOS drum machines I’ve used in the past.
• Tapping out the bass-line took a little more time than the drums. I didn’t have anything “standard” in my head, so I doodled a bit. This is where the “takes” gizmo in GarageBand came in really handy. Record a bunch of things, decide which one is most interesting, clean it up a little, throw an effect or two on it to make it more interesting, on to the next track.
• In retrospect, lots of effects is maybe a crutch. I don’t have enough taste yet to tell.
• With the drums and bass down, it’s time to adorn the track with a melody or interesting hit for effect. I added one subtle thing, but couldn’t think of anything I liked that was worth making prominent. If I were actually trying to use this beat for something, I’d keep digging. But for my first or second beat, it’s not a big deal.

I wanted to jot down my thoughts because I’d like to write more about making and understanding music, but also because I keep meaning to write down what I find challenging and interesting as I start from a “beginner’s mind” in some craft or skill. And so I did.

You’re six hundred words into this thing now, so I’ll reward you, if we could call it a reward, with “An Beat”.

Crafting your editor lightsaber

Vim: revisited, on how to approach Vim and build your very own config from first principles. My personal take on editor/shell configurations is that its way better to have someone else maintain them. Find something like Janus or oh-my-zsh, tweak the things it includes to work for you, and get back to doing what you do. That said, I’m increasingly tempted to craft my own config, if only to promote the fullness and shine of my neck beard.

Uptime all the systems

Making the Netflix API More Resilient lays out the system of circuit breakers, dashboards, and automatons Netflix uses to proactively maintain API reliability in the face of external failures. Great ideas anyone maintaining a service that needs to stay online.

List All of the Riak Keys, on the trickiness of SELECT * FROM all_the_things-style queries in Riak, or any distributed database, really. The short story is that these kinds of queries are impractical and not something you can do in production. The longer story is that there are ways to work around it with clever use of indexes and data structures. Make sure you check out the Riak Handbook from the same author.

A little bit of Clojure

Introducing Knockbox introduces a Clojure library for dealing with conflict resolution in data stored in distributed databases like Riak. If you’re working with any database that leaves you wondering what to do when two clients get in a race condition, these are the droids you’re looking for. I would have paid pretty good money to have known about this a few months ago.

Clojure’s Mini-languages is a great teaser on Clojure if, like me, you’ve tinkered with it before but are coming back to it. This is particularly useful if you’ve seen some Lisp or Scheme before, but are slightly confused by what’s going on with all the non-paren characters that appear in your typical Clojure program. Having taken a recent dive into the JVM ecosystem, I have to say there’s a lot to like in Clojure. If your brain understands static types but thinks better in dynamic types (mine does), give this a look.

—

I occasionally post links with shorter comments, if you’d like a slightly more-frequent dose of what you just read.