The Spike

I was spiking on Redis recently. I wanted to use the redis-objects gem to simulate a shopping cart app even though the README specifically says

Just use MySQL, k?

I wanted to see what would happen if I tried it anyway. So the README and examples for the redis-objects gem are great so I’m not going to rehash what’s there. However, I will say though that the example has you hardcode the id field to 1. That detail snuck up on me.

If you don’t set an ID then you can’t work with a redis-object instance. You get an exception: Redis::Objects::NilObjectId: Attempt to address redis-object :name on class User with nil id (unsaved record?)

It’s basically trying to tell you, “hey, save the record first or set an ID”. Well, honestly, I don’t want to set an id myself. This is where the meat of the README is. Redis-objects really fits organically in an existing ActiveRecord model. That means Rails. In this case though, I don’t want an entire Rails app. I can see the value though in a plain old Rails app. Just look at the examples if you want to see more.

Anyway, continuing on with the spiking, I tried to integrate the Supermodel gem with Redis-objects. That sort of worked. You just class User < Supermodel::Base and you can sort of get it to work. This is great because Supermodel gives you finders like User.find_by_email('bob@yahoo.com') to make it act like ActiveRecord but you can't use .create(email: 'bob@yahoo.com') to begin with because of the same errors as I mentioned above. Redis-objects really wants the record to have an ID already. Even using Supermodel's RandomID mixin didn't work. The initialize order and callback hooks don't really work (or at least I couldn't get them to work).

Finally, I tried combining just redis-objects and datamapper redis. That worked. And it's pretty nice. Check it out.

require 'redis-objects'
require 'dm-core'
require 'dm-redis-adapter'

DataMapper.setup(:default, {:adapter  => "redis"})

# you would move this to a common location
Redis.current = Redis.new(:host => '127.0.0.1', :port => 6379)

class User
  include Redis::Objects
  include DataMapper::Resource

  # datamapper fields, just used for .create
  property :id, Serial
  property :email, String

  # use redis-objects fields for everything else
  value :disabled
  value :name
  list :cart, :marshal => true

end

# absolutely need this line for dm-redis
User.finalize

So using this is pretty easy.

u = User.create(email: 'test@test.com')
u.name = 'Testy McTesterson'

When you look at Redis, the keys are already composited for you and magic has happened.

redis 127.0.0.1:6379> keys *
user:test@test.com:name

redis 127.0.0.1:6379> get user:test@test.com:name
Testy McTesterson

Yay!

The name field is from redis-objects and the create uses datamapper. This is a really odd pairing but I like the fact that I have no sql database in the mix but still have finders similar to an ORM. Something to keep in mind, datamapper's finders are a bit different than the Rails 3 ones (no .where method).

Benchmarking A Million Things

Ok fine. So maybe this works, maybe it doesn't. Maybe it's not the right idea. What about the good stuff? Like, how fast can we load a whole lot of names into MySQL versus Redis using the above code and techniques? Is it even relevant?

Summary
-------------------------------------------------------------------------------
(PL = pipelined redis operation)
 
Loading one million random names (full names) like John Smith, Patty Gerbee Sr)
MySQL:                   06:05
Redis:                   02:45
Redis C ext              01:32
Redis pipelined:         00:56
Redis pipelined C ext:   00:19
Ruby just loading array: 387ms
 
 
Loading 10k ecommerce-style data (orders, users, products)
MySQL:    00:09.40
Redis:    00:14.50
Redis PL: 00:02.72

A gist of these test results is here.

A More Complete Example

If you know the ID and don't need something like an auto-incrementing column outside your code/control then you can greatly simplify the code above by getting rid of Datamapper. You can simply use redis-objects to fake an ORM. I had great success using it as long as you USE NATIVE REDIS TYPES. Listen to the redis-objects author, don't try to force the tool into the use case.

# What if we want to use redis-objects as a database but
# try to stick with native redis objects?
#
# For example, Supermodel is a great gem but using the Redis
# mixin causes Supermodel to serialize to JSON strings in Redis
# which is going to kill performance.  You have to model your
# problem using native Redis objects to keep the speed up.
#
# At the same time, I miss the finders from ActiveModel
# like: Person.find('Joe')
# Supermodel does give you those finders so you will feel right at
# home coming from Rails.  I tried using ActiveModel mixins with
# redis-objects but it didn't work for me.
#
# I found the below a nice compromise but it requires a lot of
# custom methods.  :(


require 'redis-objects'

class Person
  attr_reader :name
  alias :id :name

  include Redis::Objects

  def initialize name
    @name = name
  end

  def self.exists? name
    # Here's a big assumption, if the id attribute exists, the entire
    # object exists.  This might not work for your problem.
    self.redis.exists "name:{#name}:id"
  end

  def self.find name
    # new behaves like find when a record exists, so this works like
    # find_or_create()
    self.new name
  end

  # native redis attributes with redis-objects
  value :age
  list :favorite_foods
end

# example usage

joe = Person.new 'Joe'
joe.age = 34
joe.favorite_foods << ['cake', 'pie']  # it will flatten arrays, don't worry
joe.favorite_foods << 'bacon'          # or you can do this

Person.find('Joe').age = 56

# find and initialize
Person.find('Stan').age = 21

# you cannot just .favorite_foods = ['Steak]' because that's not how native
# Redis objects work
Person.find('Stan').favorite_foods << 'Steak'

# deleting a field
Person.find('Stan').favorite_foods.del  # notice it's .del and not .delete (del is the redis cmd)

A musician practices their art constantly. An athlete trains. And yet, we in the tech community have to go out of our way to find people who work on personal projects on the side or have the will/motivation to learn new things outside of “the job”. If you don’t learn on your own then you have to figure it out as you go. And you are already very busy so I know you’re just skimming over this post anyway. So let’s break this down in a bulleted list.

• Developers work best when they’re challenged but not overwhelmed. Optimal work is a run, not a jog or a sprint.
• Developers will step-up or fill-in a task to make it challenging as stated above.
• Your classic boss doesn’t really care how you solve problems.
• Culture takes 20 years to change.
• The Blub Paradox says (among other things) that you can’t make anyone understand the power of a different language because developers become sedentary.
• Good consultants are already busy so no one is going to save you.
• Training or bootcamps won’t change your skillset or habits by a large percentage.

Let me put this another way.

My cat will play with a toy on the floor. And after a while, it becomes boring. So she swats the toy under a cabinet. And now it’s exciting! She can barely reach the toy and its a challenge. When she gets the toy out, she’ll bat it under the cabinet again. What is she going to do? Be bored again? She’s good at reaching for the toy and using her claws to hook stuff, tail counter-balance to stay in control and seeing in dark places. She needs to be playing under the cabinet and not given an mouse in a barrel. This is what you must do. You must improve your skills by going beyond “Good enough”. Otherwise, you will stagnate and have limited tools and approaches to problems.

But how do you even know what to learn? When you are looking up and down the power continuum in a problem domain, how can you grow beyond your own knowledge boundaries to see advantages that other people are enjoying? If you were looking at a menu of pies, how would you choose which one you will like?

Here’s the pie menu. Pick the one that you like!


PIE MENU
(all pies come with whipped cream on the side)
---------------------------------------------
Cranston Ermu Supreme Pie
Tagasnackle Mound Blat Pie
Rumination Flip Pie
Rainclouds and Humility Pie
Are You Seeing The Problem Pie

$3.99 per slice


What? This is terrible. This isn’t fair. Certainly technology cannot be like this. This is computer science! How can you choose? How can you know?

Hungry: “Waiter. What’s in humility pie?”
Waiter: “Humility.”
Hungry: “What does that mean?”
Waiter: “Have you ever literally tasted humility?”
Hungry: “No.”
Waiter: “Well it’s kind of like tasting regret but also different than pride. You kind of taste like you should have prepared more or set expectations lower.”
Hungry: “That doesn’t make any sense.”
Waiter: “Of course it doesn’t. It’s an experience. It’s not a definition.”

Given enough questions, the Waiter will quit unless the Hungry customer chooses a pie or leaves.

Now let’s look at some horrifyingly contrived personalities looking down (or not looking down) the pie continuum.

The Eternal Procedural Coder

Let’s say you are a developer who just writes scripts all the time. You piece things together and when they work the first time, you ship it (whatever shipping means to you). If a more complicated task comes along, you don’t change your design, tools or strategy, you just change the amount of time you throw at it. Once it works, you call it done.

Exactly what is the exit strategy here? What’s the end game? Who or what is going to save you or change your situation? What would the disruption look like? What would the disruption be called?

Meanwhile, you hate looking at old code. You wonder “what was I thinking?” or “I must have had a lot of coffee this day”. Projects come and go, you hope you never have to go back to old projects. You leave things alone because they are terrible but do the job. You continually start new tasks but they all come out the same. Huge single files, huge methods/functions. One mess after another. No wonder why they call this “work” right? Who would do this for fun or learning?

What’s interesting is that The Blub Paradox applies here too. It’s difficult to explain what development is supposed to be like to someone who has never had a success. Success breeds success. Failure is of course crucial to learning and courage is needed to fail when trying to succeed. But experience really is the success side of trying and learning. You can gain experience through failure but in terms of visualizing the ideal, success is very important.

The It Can’t Be Done Guy

Let’s say there’s a website called Searchbox.com that’s a stripped down version of Google. In a meeting, everyone is talking about how Facebook search and Twitter are all hot right now and you feel like you are behind the curve. So you decide you want to add friends, followers and other social features to Searchbox.com. Your rockstar lead developer says “it can’t be done”. No one questions him. The idea dies, jokes are cracked and the grapes were probably sour anyway. Yay team?

Software is supposed to be soft. So why can certain things be so easy and other things be so hard? I understand certain problems just “are hard” (concurrency, cache invalidation). But not even entertaining ideas or experiments could be a sign of bad design. Even with crappy procedural scripts, simple changes mean omfg-lots-of-lines changed. Design isn’t about avoiding change (and change will happen), it’s about reducing the cost of change.

If you want to learn from a master, read POODR. Sandi says:

…successful but undesigned applications carry the seeds of their own destruction; they are easy to write but gradually become impossible to change.

POODR is full of these insightful observations and Sandi doesn’t just give you advice. She walks you through refactoring code and demonstrates the concepts that are related to these insights.

Anyway, back to our example project. If searchbox.com is broken up into modules then there is a user class or object. Surely this class could be forked into a feature branch and worked on in isolation by a few team members to add Friend and Follower traits/mixins/interfaces? Or is the user really a session cookie combined with a database row and “who knows” how to change it?

If there isn’t really a concept of a user but it’s just a bunch of hacks then yes, no one is going to entertain massive changes to this concept of a “user”. It’s too hard (in your system). However, if a system was explicitly designed thoughtfully (not by default or by accident) then features are easy to add. The software stays soft. This is what Sandi is talking about in POODR.

The Waterfall by Default Guy

Let’s say there’s a guy in charge of a project. He hasn’t ever tried Agile of any sort. He’s read about it and he’s confident that he knows all about it. He thinks it sounds crazy, lazy and chaotic. He’s done plenty of projects in the past that have come out under-budget and on-time so why change anything? On the other hand, he’s not technical so his opinion of past projects is completely dependent on the feedback given to him by developers. If developers ever felt like talking to him was a waste of time then his opinion of how good “waterfall” is could be slightly off. The real question is, can you explain how delicious Agile Pie is to him?

Here’s the wheelhouse of agile type stuff.

• Developers aren’t going to want to work with you again if you burn them out.
• Project heroes burn out.
• You can’t hire your way out of a failing project.
• Most people don’t like the term Waterfall because it’s viewed old.
• If you don’t declare a methodology it’s Waterfall by default.
• You can’t do Agile. Agile is a type of thing. Like Car. You have to buy a type of Car.

So when you try to explain any type of agile techniques to someone who doesn’t have any experience with it, it’s the same thing as the Blub Paradox. The Waterfall Guy is happy enough doing no explicit methodology, why would he change? When asked how to prevent projects from getting off-track he will say, “before you start working on a project, you need to know what you are building”. Even though Agile (and common sense) says that both the customer and the developers are the stupidest at the beginning of a project. Big Design Up Front (BDUF) doesn’t work. BDUF is undesigned project methodology. BDUF is how you put together simple things like grocery lists and legos. BDUF isn’t how you deal with change and complicated systems where developers will upscale the project difficulty until it becomes interesting.

So even if you discuss these items with Waterfall Guy, he’s not going to get a taste of Agile Pie without eating it. You could talk all day long about the ingredients, how people keep eating it, how there’s these case studies showing it’s delicious. “Meh, whatever. I have my dessert already.” At the same time, Waterfall Guy is always asking for visibility as to what developers are doing through status reports. In his heart he knows that he has a few hero developers and if he loses them, he’s screwed. That’s ok, that’s why they call it work right?

The Non-Tester

There’s a team who are running a production app. They have a testing team. When developers are working on “omg the really important thing”, they tell the testing team to run “whatever test”. It usually involves someone manually going down a list with a pencil. When a problem is found then a tester will tell a developer something went wrong. “It showed me this error 0×2123 when I did the thing you told me to do.”

Except a lot of the time there isn’t anything wrong. A lot of times the problem is the testers aren’t developers. Otherwise, they’d be developers. A lot of times the tester doesn’t really understand the system and the developer has to explain the system to them. So while the developer is working on “omg the next really important thing” the tester is asking questions and the developer is getting annoyed. Jokes are made “you only come to me when there’s a problem, haha”. Or, “I bug you so much I owe you a beer! Haha.”.

Looking down the power continuum, a developer looks at testing libraries. They look annoying and time-consuming. Who needs this hassle? They don’t have time to find out if they do. They don’t understand the different levels of testing (from most involved and beneficial):

• No testing at all – you have to hit refresh in a browser and you have no idea if you broke anything else. Old bugs pop up.
• Some testing – unit tests but you have a team run through end-to-end scenarios.
• A lot of testing – you have unit and integration tests and maybe even code coverage reports and CI.
• Test-first development – you write your tests before code but that’s the end of that methodology.
• Test-driven development (TDD) – you let your tests completely drive the design of the system.
• Testing end game – You do red-green-refactor. Your tests run fast. You do UI testing. Your customer requirements map to easily read executable stories. You spend very little time in red. You ping-pong pair with TDD. No code is committed without passing CI. Your testing tools and process are constantly evolving in response to pain points.

The non-tester doesn’t see the value or the constant effort people are making to try to get to the Testing End Game. “I’m not a tester! Why would I test!” Meanwhile, they complain about how little time they have because they have to manually have to see if their code works. Or maybe they are embarrassed / mad / frustrated that they broke the development/production build. In essence, the Blub Paradox is telling us that they can’t see their problems through the lens of Blub because they don’t know Blub. Meanwhile, Blub (TDD) practitioners can’t imagine doing it the way the non-tester does it. In fact, TDD practitioners are honing their craft so much that it is a rare event when they’ll take the time to explain TDD or the End Game to a non-tester. Even if they do, both will be frustrated because any insight or evangelizing will be misunderstood.

The Long Time Ignorant

Take any of the above and put it in the context of time. How long has SQL been around? Does someone on your team not know SQL? Is it because they haven’t needed it? What if they are writing flat database files by hand and do need SQL? Have they just been living under a rock or can no one change their mind? What can you do as an individual to change such a blind eye?

How long has JUnit been around? Is there a Java developer you know that hasn’t written a unit test? Or they have written a few but they don’t anymore because they think they don’t have time to do it?

That Seems Hard

Someone who can only write code in a procedural style can’t break down problems into easily solvable bits. Without concepts like Mixins/Traits, Composing Behavior and TDD; a complex problem is just going to seem too hard. “I don’t do that kind of stuff.” When someone that has OO or Functional experience might say “here’s how I would do it, who wants to help me?” Of course there are things that are just too big but the difference is in experience. A Bash scripter isn’t ever going to understand ncurses events because they haven’t ever written a desktop GUI. So even though ncurses can be great for turning “scripts” into “programs”, they are going to shy away from ncurses because “wtf that seems hard”.

• You can’t explain the benefits of a web framework because of the Blub Paradox.
• You can’t explain the benefits of an ORM vs raw SQL because of the Blub Paradox.
• You can’t explain the advantages of tmux to someone because of the Blub Paradox. Even while they keep losing their ssh sessions over unstable wifi. Given a simple enough problem, you might be able to convince them to try it out.

Tmux is a perfect example. No one understands how great it is until they use it. Then and only then do they never want to go back. Tmux isn’t always great though. Sometimes the terminal gets all weird with certain keyboards or maybe you want the native buffer scrollback to work. So it’s not some silver bullet default. However, once you grok tmux, you know when to use it and miss it when you don’t have it. This is true for many things that people evangelize. However, sometimes you need to understand their world view and take that into consideration.

Tmux pie is delicious. Ask anyone who has tried it.

Answers

I don’t have all the answers. Ok, see you later. *silence* Well listening to Rubyrogues and reading POODR has given me some memorable advice which I’ll repeat here.

• Spike on foreign concepts. Ok, you don’t know OOP and breaking down problems. Make up a problem and do it. Don’t try to learn in your main project. Spike on a concept and throw the code away.
• Pair with someone who knows this stuff and check your ego at the door. When you are lost, don’t play with your iPhone. Ask questions. Pretend to be stupid. Check your ego.
• Learn a different tech stack. When I write Java (poorly these days), I bring Ruby experiences back with me.
• Watch conference videos and screencasts. Pre-learn before work. You have no choice. No one is going to pay you to get better.
• Read the POODR book even if you’re not a Rubyist.
• If you are resistent to an idea, change sides. Let’s say you don’t like soccer. Take the position of a soccer fan and argue with yourself. You might be amazed at the argument you make just looking from another perspective.

I was playing around with the falcon p385 patch to see if it’s any faster than some of the more recent MRI rubies.
TL;DR version: looks like p192 is faster than p385 of any type or tweak.

Here’s how to get a p385 Ruby version patched with funny falcon’s performance patches using RVM.

mkdir ~/.rvm/patches/ruby/1.9.3/p385
curl https://github.com/funny-falcon/ruby/compare/p385...p385_falcon.diff > \
$rvm_path/patches/ruby/1.9.3/p385/falcon.patch
rvm install 1.9.3-p385 -n perf --patch falcon

Then rvm use 1.9.3-p385-perf or set it as your global ruby.

Test Setup

The following benchmarks were run on an i7 server with a RAID5 array. The disk is slow (lack of large cache) but the benchmarks were run on the same box so it should compare apples-to-apples.

From here on out, here are the definitions for the Ruby versions.
p194 = 1.9.3p194 default
p385 = 1.9.3p385 default
falcon = 1.9.3p385 with the above falcon diff patch applied
gcc_tweak = the falcon patches with GCC compile flags tweaked.

So what are these GCC tweaks? Explicitly setting the CFLAGS for your machine’s CPU type and recompiling ruby with the Falcon patches applied.

Micro and Macro Benchmarks

I used the ruby-benchmark-suite to run these tests.

Here are some example results. I can’t list them all. There are over 100 benchmarks. These are results for the mean times in seconds.

Here’s the winner summary for mean times. This is the number of times the ruby version was the fastest for a particular benchmark.
1.9.3p194 – 64 wins
1.9.3p385 – 29 wins
1.9.3p385-falcon with GCC tweaks – 10 wins
1.9.3p385-falcon 5 wins

Boot time and IO

Timing rails boot time is a bit more important to me. If you want to know how to really save “rails boot time” see the DAS screencast on not loading Rails at all.

Even when using domain objects and lib tricks, it’s nice to have Rails and all I/O boot fast. The main thing that funny falcon’s patches do is speed up requires and I/O.

So let’s benchmark booting a Rails app.
$ time bundle exec rake environment

In conclusion, p194 looks faster on the macro and micro benchmarks but Falcon patches boot Rails faster.

As I’ve previously talked about, Hashes of Hashes are weird to work with. In the previous post about Captain Planet, I showed how to select, filter and manipulate 2D hashes and arrays but ultimately concluded that a hash of hashes is both weird and unnecessary (most of the time).

If you can control the data, inline your key into the hash data and make an Array of Hashes. It’s really where it belongs. If you don’t, you’ll find yourself doing a few extra iterations or work. Below you’ll see a simple example of the two data structures.

# Hash of Hashes
{
  :dave => { :age => 1, :height => 2 },
  :tony => { :age => 1, :height => 2 }
}

# Better --> Array of Hashes
[
  { :id => 'dave', :age => 1, :height => 2 },
  { :id => 'tony', :age => 1, :height => 2 }
]

In this case of an array of hashes, the data is easier to manipulate with array operations and filters. But before talking about matching on arrays of hashes, I want to talk about matching on dates.

doc = Date.parse "Nov 12 1955"
marty = Date.parse "Oct 27 1985"
future = Date.parse "Jan 13 2094"
doc < marty
=> true

So comparisons with dates work like you might expect. But fuzzy matches do not. Take this example. Is Marty in between Doc Brown and the distant future? The answer should be true.

We use a Range (x..y) object to make a date range. Then we can use === to check if we get a match.

marty === (doc .. future)  # wrong, wrong, wrong
=> nil

But wait, === on Date is different than === on Range. The method === on Date checks to see if it’s the same Date whereas === on Range checks to see if the argument is within the range. So if you flip it, it returns true.

(doc .. future) === marty
=> true

require 'date'

holidays = {
  :halloween => { :date => Date.parse("Oct 31 2012"), :presents => false },
  :christmas => { :date => Date.parse("Dec 25 2012"), :presents => true },
  :july_fourth => { :date => Date.parse("July 4 2013"), :presents => false },
  :valentines_day => { :date => Date.parse("Feb 14 2013"), :presents => true },
  :thanksgiving => { :date => Date.parse("Nov 28 2012"), :presents => false }
}

# turn hash of hashes into array of hashes
holiday_array = []
holidays.keys.each do |key|
  holiday_array << { :id => key }.merge(holidays[key])
end

# find all the holidays with presents
puts "yay presents!"
puts holiday_array.select {|holiday| holiday[:presents] == true }

# find all the holidays within a date range
winter = (Date.parse("Dec 21 2012")..Date.parse("Mar 20 2013"))

puts "Winter holidays"
puts holiday_array.select {|h| winter === h[:date] }

So what we did in the middle there was flatten the hash of hashes into an array of hashes by merging the key with the ‘data’ part of the hash. Hopefully that makes sense.

Listen to Dave Thomas

Watch this video -> please. Start at 45:50 and watch at least 10 minutes.

I tell a lot of people about this video. I don’t tell 10,000 people up on a stage on some world tour. But whenever I get into a good discussion with geeks, I reference this video and tell them to watch it. So I tell a lot of people in terms of percentage in conversations that I’m enjoying. So in other words, I really like the video personally, it struck a chord (or a brain-thorn) and I want other people to consider his point of view. Specifically, Dave Thomas says you should learn a new language every year. Invariably, I’ll tell people that Dave Thomas writes books and a few times people will dismiss his main idea because they think he’s selling books.

Dismissing Dave Thomas’ opinion that you should learn a new language every year just because he is an author is a crap position to take. First, FUD. Second, Dave Thomas doesn’t sell books about the languages he says to check out in the video. Thirdly, I trust Dave Thomas. This isn’t some fucking Larry Ellison power play.

So in terms of following his advice, I might pick up Scala. It’s different enough and anti-hipster-omg-embrace-the-java-enterprise enough that it will challenge me and/or be different than say continuing with Python (will is really Ruby like Spanish to Portugese) learning. I’ve picked up a Scala book but haven’t started it yet. I looked at the Lift docs and was completely deflated. Chad Fowler tweeted about the Play! framework (thanks for the reminder) and thank god this looks better.

If I get sick of a city, I might move. When I move, I’m looking for something new. Similarly, starting out with Scala is going to be a bit like jumping into a new town. Ok, I need to find the equivalents of the old things that I’m used to. But wait, I moved to a new town to do something new! If I bring my old habits with me, I’ll just be writing Ruby code in Scala. And that’s crap.

Marge: I’ve dug myself into a happy little rut here and I’m not about to
hoist myself out of it.
Homer: Just bring the rut with ya, Honey.
– “You Only Move Twice”

What any technology stack is going to have to do on our first date

I need a list of libraries at least searchable by category and popularity. Popular gems are likely to be used and therefore good. This is great to get to higher level operations and gain productivity. For example, ruby-toolbox.com.

I need screencasts ordered by date so you can learn community trends. I’m going to need a way to keep up with the Jones’ and know what’s falling out of favor in the community because a shiny new library, technique or concept is getting the attention of key people. This also operates as curated content. For example, everyone is talking about CORS. I’m sure there will be a Railscast on CORS even though CORS is not specific to Ruby or Rails.

Related to that last point, the community should be of high quality. I know people like to hate on PHP but if you took a core sample of PHP-land, I think you wouldn’t be too happy. Of course there are exceptions, for example, Facebook is full of brilliance.

Some easy way to install things that’s Internet-enabled. Maven is close. Rubygems is good. NPM is good. Yum beats plain RPM. Pip is fine. CPAN is crap. All these package managers connect to the Internet and resolve dependencies. As a bonus, being able to repackage, bundle, freeze or tar up libs into a lib directory is great for corporate firewalls or making deployment not suck so hard.

I need a way to separate projects from one another. Gemsets with RVM does this. Virtualenv does this with projects. Lib implicitly does this in Java (although CLASSPATH issues suck). Explicit project isolation with libs is great.

Give me videos from the community like Confreaks. Video community talks are necessary. The JavaOne video site sucks and only because of the content. It doesn’t have to be flashy and overly-produced. Oracle publishes 3 minute videos with zero details and zero content. Yay?

I need quick feedback. This means a REPL and/or event-based test suite runners. If I save a file, I want a test to run (Ruby’s Guard gem). If I want to figure out how to do something, I want it to be instant feedback. About the farthest thing I can think of away from this is deploying to Tomcat, reloading the webapp and hitting refresh in a browser (#fml).

I’ll call Scala back

So far, Scala has the REPL thing down (just type $ scala). And the SBT tool does auto-reloading similar to Guard. Of course, it has to compile things so it’s slightly a slower feedback loop but it’s not terrible. SBT also seems to have a manifest thing going on for the equivalent of gemsets. It all goes in project/plugins.sbt which is great. There’s also a concept of a global project in a ~/.sbt folder. This reminds me a lot of NPM.

I haven’t discovered the community side of things yet like screencasts (railscasts) or a news site (rubyflow). On the negative side, there seems to be some Java crosstalk. For example the Play! framework can generate a Scala or a Java project. So if you are trying to get away from Java, this is a good step away but not a clean break. I get a feeling of Java refugee camp (which is fine).

Something I’m working on right now is understanding the IDE landscape. Eclipse was a bit weird with loading the Scala plugins and Intellij is coming out with a new version of IDEA soon. Also I need to grok the sbt-console inside the IDEA plugin vs command line. Are they the same? I had this same challenge when trying RubyMine. At some point you have to get a feel for what the One True Way was intended to be.

Zeus

Video of Zeus in action. Is it the new spork?

Super fast Rails boot times, but not just booting. Everything! Like DRB on steroids.

FishFish shell

`brew install fishfish`. The new ZSH? A fork of the Fish shell. Featured in the above video link. Be sure to uninstall fish through homebrew if you have already. It’s not the same thing.

YADR

YADR – for me, the new codegram/vimfiles? Impressive documentation, will really change my muscle memory.

Prezto

Included with YADR, looks to be a replacement for oh-my-zsh (my fav). Will have to try it out and update this.

Anyway, I wrote a script that does this:

• Creates a repo
• Unzips Linux 3.6.1 source
• Add all files / commit all files
• Branch / Modify / Commit / Merge (3 times)
• Cleans up branches, cleans up temp directories

The Linux kernel source is 79MB, ~42,000 files. So this is a bit excessive. You probably won’t be working with a project this large. However, it’s a good way to show speed and performance.

The other interesting thing that I found while doing this test is how much longer my SVN script turned out to be. Both scripts do the same thing but the Git script was much shorter. So in a way, Git seems to be faster and easier (or at least less typing) at the same time.

A note about the benchmarks. The Raid 5 test was done on a web server with a Raid controller that has very little cache. No question that these spinning disks are a bit slow under load. However, the tests were done on the same box so it’s still a fair comparison. The SSD row is a virtual ubuntu machine on an SSD; a different machine all together.

Benchmark Times mm:ss

Git has a bit of an advantage by doing 3 branches because it does its garbage collection only once on the first merge. So I apologize for the bias. I can say for sure that the one branch test was something like this: SVN 20min, Git 2min.

Here are the links to the scripts I wrote if you want to try them yourself.
svn_stress.sh
git_stress.sh

Run the scripts with the time command:

$ time ./svn_stress.sh /tmp/stress_test
233.78s user 81.81s system 35% cpu 14:56.48 total

After decompressing and reflecting, I came to a philosophical realization.

• Rubyists were not taught Ruby in school or elsewhere, they willed their skill into existence.
• Internet access came through the air over cell and we suddenly had dev environments in the middle of the woods.
• The wikipedia description of game of life was just some english bullet points that were implemented 100 times

So in these three ways, we willed things out of the ether and that is awesome.

For a less philosophical run-down of what happened, check out Brandon Hays’s blog.

===============================
= Fri Jan 1 21:46:13 EDT 2034
===============================

Too long. So here’s what you can do. Go to Bundles->Bundle Editor->Show Bundle Editor. Then go to the Source category. You can filter at the top for Snippets if it’s easier. See this picture.

Then put this code in:

${TM_COMMENT_START/\s*$//}==${1/(.)|(?m:\n.*)/(?1:=)/g}==${TM_COMMENT_END/^\s*(.+)/ $1/}
${TM_COMMENT_START/\s*$//}= ${1:${TM_SELECTED_TEXT:`date +'%a %b %d %Y'`}} =${TM_COMMENT_END/\s*(.+)/ $1/}
${TM_COMMENT_START/\s*$//}==${1/(.)|(?m:\n.*)/(?1:=)/g}==${TM_COMMENT_END/\s*(.+)/ $1/}

This will give you a banner like this:
===================
= Fri Jan 1 2034 =
===================

And it’ll be the right length automatically. Nice.

Caching in Rails is really simple (straight from the docs):

You can set up your application’s default cache store by calling config.cache_store= in the Application definition inside your config/application.rb file or in an Application.configure block in an environment specific configuration file (i.e. config/environments/*.rb). The first argument will be the cache store to use and the rest of the argument will be passed as arguments to the cache store constructor.

Originally, I didn’t even know it was there. So I was thinking I’d just create a hash in memory. So I wrote my own cache class like this (homegrown_cache.rb):

# not exactly a pure cache, implements the lookup or expensive operation too
# probably needs to be decoupled

class Cache
  def initialize
    @cache = {}
  end

  def cache(name, values = {})
    if values = {}
      return @cache[name.to_sym]
    else
      @cache[name.to_sym] ||= values
    end
  end

  def search(name)
    if @cache.keys.include?(name.to_sym)
      puts "getting name from cache instead of expensive operation"
      return @cache[name.to_sym]
    else
      self.cache(name, Expensive::Search(name))
      return @cache[name.to_sym]
    end
  end

  def clear
    cache_size = @cache.length
    @cache = {}
    return cache_size
  end

  def to_s
    "#<Cache:#{self.object_id} Cache Size: #{@cache.length}>"
  end

  def size
    @cache.length
  end

end

Then you might want to use it in an implementation like like CacheEnabledSearch or something like that within your app.

But it’s way easier and smarter to just use the goddamn caching that’s built into Rails. Specifically, if you use the in memory store, since Rails 3.0 it’s thread safe. Meaning if two clients hit your cache, you’re not going to have weird bugs pop up. It’s very easy to configure and use.

I’m posting the Cache class anyway though it’s interesting how simple an object it is without being thread-safe. But even then, I’d recommend using the cache class from Rails outside of Rails.

So here’s how you’d use Rails caching “without” Rails. In other words, a console app.

# using the Rails caching class outside of rails
require 'active_support'

# 
cache = ActiveSupport::Cache::MemoryStore.new
cache.write('user', 'bob')
# => true
puts cache.read('user')
# => bob

Not particularly interesting. This would be way more useful inside of a Rails app where a singleton class called Rails.cache is set if you have something set in your config:
config.cache_store = :memory_store

Let’s avoid the Rails situation for now and create a file cache directory for a file based cache:
$ mkdir /tmp/poro_cache

Here’s a complete example.

# using the Rails caching class outside of rails
require 'active_support'

cache = ActiveSupport::Cache::FileStore.new "/tmp/poro_cache"
if cache.read('user').nil?
  cache.write('user', 'bob')
  puts "wrote to cache, run me again"
else
  puts "Reading from filestore cache: #{cache.read('user')}"
end

If we run this one time, we get this:
ruby rails_cache_poro.rb
wrote to cache, run me again

And running it again:
ruby rails_cache_poro.rb
Reading from filestore cache: bob

Reset it with rm -rf /tmp/poro_cache

Ok, so that’s ActiveSupport caching outside of Rails. Wiring Rails up to do this inside a controller, lib class, helper or other ruby object is trivial once you have followed the caching guide. You won’t instantiate ActiveSupport::Cache anymore directly but you’ll do it indirectly through the config block for your environment.

And then all the statements will look like this:

>> Rails.cache.write 'foo', 'bar'
=> true

But it works the same. Really easy. Of course, the next subject which is more complicated is when to invalidate your cache.

I have a Windows 7 box. Normally I dev on Mac (not perfect but sucks less). I had built a Linux VM (ubuntu) but wanted to share my projects from the VM to windows to use sublime text 2. Editing through the shell is ok but I wanted a full text editor instead of vi (which I haven’t gotten my speed up on). So I normally watch my files for changes with Guard as previously blogged.

So I share out my ~/projects directory through VMWare to Windows. Great! Now I can have all the unix tools in the VM and a GUI editor without doing the console screen painting through VMWare. The problem is, libevent doesn’t trigger through the shared folder feature of VMware. I have no idea why.

So I pivoted. I tried the windows rails installer. Even though it tries to pre-patch mingw32 right before the native gem install, it didn’t work well with native libs like nokogiri (just to name one).

So I tried cygwin and apt-cyg (apt-get for Cygwin, nice, sorta). It didn’t work well with compiling ruby even after googling flags and installing gcc. Was getting “could not find C compiler”.

So this journey continues. I installed the latest version of VMware workstation on Windows, going to try shared directory again with new vmware version and ubuntu 12.04. Maybe there will be some magic in the upgrades. I doubt it but I will try.

I could also try my inline rspec trick even though that still uses libevent for triggering the test execution.

If you have no idea what I’m talking about but am curious, leave a comment and I’ll make a video or follow-up.

Update:Doh! You can just do text.split(/(\.)\s+/) to make the regex keep the delimiter with the parens group match. Then you have to join in groups and some other stuff. I keep the post below for posterity. :)

When you split a string on a character, you get an array back without the split character. When splitting sentences, the period goes away with the split. So we can employ some rather nasty behind regex tricks to remember the puntuation and still do the split. Pretty great!

Here’s some code I stole from stackoverflow that splits sentences fairly well. Pay no attention to the complexity of this unchained beast.

# oniguruma regex to split on multiple sentence
# delimeters but preserve delimeter character.

text = "Hipsters are everywhere! Even in Home Depot? Flannel wrench set in ma face."

result = text.split(/((?<=[a-z0-9)][.?!])|(?<=[a-z0-9][.?!]"))\s+(?="?[A-Z])/)

puts result

When you run it:

Hipsters are everywhere!
Even in Home Depot?
Flannel wrench set in ma face.
[Finished in 0.3s]


That’s great but it does a fat lot of good when trying to modify it to reverse sentences. Not to mention, regexes falling down on edge cases (the above doesn’t work for shouting sentences like this one?!!

Ok, whatever. That’s regex stuff. Let’s try applying what we’ve learned from our last post and create an inline rspec test to do what we want.

First, we’ll test the simplest case with See spot run. And then we’ll add multiple sentences in.

# oniguruma regex engine testing to split on multiple sentence
# delimeters but preserve delimeter character.

require 'pry'

class Reverse

  def sentences(text)
    split_everything = text.strip.split(/(\.|\?|\!)/)
    sentences = split_everything.each_slice(2).to_a.map {|pair| pair.join}

    sentences.collect do |sentence|
      punc = sentence[-1]
      sentence.chop!
      sentence.downcase!
      words = sentence.split(" ")
      reversed = words.reverse.join(" ") << punc
      reversed.capitalize!
    end.join("  ")
  end

end

describe Reverse do
  it "reverses words in one sentence" do
    text = "See spot run."
    reversed = subject.sentences(text)
    reversed.should == "Run spot see."
  end

  it "reverses all sentences in some text" do
    text = "See spot run.  Run spot run.  Your soul is mortal spot."
    reversed = subject.sentences(text)
    reversed.should == "Run spot see.  Run spot run.  Spot mortal is soul your."
  end

end

Works pretty well. We can keep adding edge case tests to handle weird punctuation situations or the dreaded “Mr. Smith goes to college!” sentence.

In rails, I’d just put a binding.pry and magically, I’m at the breakpoint to play around at that point in time. No copying and pasting. Nice.

After watching a bunch of destroyallsoftware screencasts though, I like his one setup from episode #7 where he inlines rspec tests. Combine this with watchr and I might have a new favorite.

Let’s say I had the following hello world example.

class Hello
  attr_reader :message

  def initialize
    @message = "Hello Variable!"
  end

  def say_hello
    puts @message
  end

end

When I run it in Textmate with Cmd+R, it doesn’t do anything. Because there’s no “main”. So then I have to add:

Hello.new.say_hello

Great! But wait a minute, when I use it in another class, it’s going to print “Hello Variable!” out. So then I take it out? Then to test it again, I put it back in? This workflow although easy kind of sucks.

So let’s just inline some rspec. But before we even do that, we need rspec and watchr. We are going to use watchr to watch our files for changes. It’s best to install rev (Linux) or ruby-fsevent (Mac) for filesystem event firing.

gem install watchr rspec ruby-fsevent

Next, run this in a terminal and put it so where you can see it run while in your editor:
watchr -e 'watch("hello.*\.rb") {|m| system("clear && rspec -c #{m};") }'

Like this:

When you save the file (even without changes), it should say:
No examples found.

Finished in 0.00007 seconds
0 examples, 0 failures

Ok, now let’s inline some rspec tests to test our class.

class Hello
  attr_reader :message

  def initialize
    @message = "Hello Variable!"
  end

  def say_hello
    puts @message
  end

end

describe Hello do
  it "has a say_hello method" do
    subject.respond_to?(:say_hello)
  end
end

Now we will get this nice rspec to the right in color immediately upon saving:

So how about irb? What if we want to learn/play in the code? Well now that we have a class and a shell instead of Textmate, we can use pry again.

require 'pry'

class Hello
  attr_reader :message

  def initialize
    @message = "Hello Variable!"
  end

  def say_hello
    puts @message
  end

end

describe Hello do
  it "has a say_hello method" do
    binding.pry
    subject.respond_to?(:say_hello)
  end
end

And now when we save, we’ll be popped into a Pry shell.

So give this a shot and see if you like this as an alt workflow.

It’s a state machine.

What huge numbers would a calculator be unable to calculate anyway? You know that scientific notation
1.274921411340194917490141234 × 10^27? I’ve seen that in a calculator. Doesn’t that let the calculator handle a number of extremely large size? Is that why it can’t play games? Does a calculator not calculate enough? Or, is a state machine that is too slow to render real-time graphics.

It’s not that a calculator can’t store big numbers. It’s that a computer is so quick on changing its internal state that it can trick us with simulations and graphics.

A CPU is not a brain either but that’s a different post.

I encourage you to checkout the State Machine gem to play around with this concept. It has direct relevance to game development and other areas.

Of course, a more elegant solution is to not pre-fork and use Gary Bernhardt’s (and others’) advice of extracting domain objects from rails and running tests over that. This is fine within plain old ruby objects but sometimes you need to load rake or factory_girl and that requires rails. So let’s assume you can’t isolate your tests into lib/ or extras/ and you need fast tests while loading your entire Rails environment. That’s where Guard and Autotest come in.

The newest Guard installs Listen which doesn’t work. So I had to back down a version of listen. Added gem ‘listen’, ’0.4.7′ to my Gemfile to get rid of polling warning. 0.5.0 is latest but it has a problem right now (bug opened 2 days ago as of today, grr). All that is unrelated to the real problem.

Spork-minitest doesn’t work with guard. The issue is that spork-minitest doesn’t support any options like -r or -e that the guard is passing.

So let’s switch to autotest. But first, let’s look at my current configuration and I’ll show you what I had to change. This is what my Guardfile looked like with Guard on a Rails project:

guard 'spork' do
  watch('config/application.rb')
  watch('config/environment.rb')
  watch(%r{^config/environments/.+\\.rb$})
  watch(%r{^config/initializers/.+\\.rb$})
  watch('Gemfile')
  watch('Gemfile.lock')
  watch('test/test_helper.rb') { :minitest }
  watch(%r{features/support/}) { :cucumber }
end
 
guard 'minitest' do
  # with Minitest::Unit
  watch(%r|^test/(.*)\/(.*)_test\.rb|)
  watch(%r|^lib/(.*)\.rb|)     { |m| "test/unit/lib/#{m[1]}_test.rb" }
  watch(%r|^test/test_helper\.rb|)    { "test" }
 
  # Rails 3.2
  watch(%r|^app/controllers/(.*)\.rb|) { |m| "test/functional/#{m[1]}_test.rb" }
  watch(%r|^app/helpers/(.*)\.rb|)     { |m| "test/unit/helpers/#{m[1]}_test.rb" }
  watch(%r|^app/models/(.*)\.rb|)      { |m| "test/unit/models/#{m[1]}_test.rb" }    
end

Delete that whole Guardfile and let’s switch to autotest with spork and minitest.

This is what my Gemfile looked like with Guard:

group :develop do
  # unrelated gems here.
  gem 'minitest'
  gem 'spork'
  gem 'rb-inotify' if RUBY_PLATFORM.downcase.include?("linux")
  gem 'rb-fsevent', "~> 0.9.0" if RUBY_PLATFORM.downcase.include?("darwin")
  gem 'guard'
  gem 'guard-spork'
  gem 'guard-minitest'
  gem 'spork-testunit'
end

This is what my Gemfile looks like when switched to Autotest:

group :test, :develop do
  # unrelated gems here.

  # spork and autotest allow you to run tests when you save a file.
  # run `spork` in one terminal from the project root.
  # run `bundle exec autotest -cf` in one terminal from the project root.
  # Then edit test files or app/* or lib/* and watch tests run automatically on save.
  gem 'spork'   # Spork caches rails so tests run fast.
  gem 'autotest-standalone' # The file '.autotest' makes sure the tests are run via test server (spork).
  gem 'autotest-rails-pure' # -pure gives us autotest without ZenTest gem.
  gem 'autotest-growl'      # growl notifications, complains a little bit if growl isn't installed
  gem 'autotest-fsevent'    # react to filesystem events, save your CPU
  gem 'spork-minitest'
end

It works great! As I said in the Gemfile comments:

• Start `spork` in one terminal
• Run `bundle exec autotest -cf` in another terminal


-c is for avoiding re-running rake test on red/green
-f is for fast start to avoid running rake test on startup


Save a file like products_controller_test.rb and watch it run. Save the app/controllers/products_controller.rb and watch products_controller_test.rb run.

The only problem is that it reacts a little bit slow because of a 1 second default sleep in Autotest. So this is my entire .autotest file which includes a hack to reduce that 1 second to 0 seconds. No impact to CPU and it reacts as fast as Guard did.

require 'autotest/fsevent'

class Autotest
  # run tests over drb server (spork)
  def make_test_cmd files_to_test
    if files_to_test.empty?
      "" # no tests to run
    else
      "testdrb #{files_to_test.keys.join(' ')}"
    end
  end
end

# tighten up the time before the test is run
ObjectSpace.each_object(Autotest).first.sleep = 0

I plan on doing a pull request to address this problem. So watch for that.

“We need to write down how it works, what we are doing and all the little tricks so that if you get hit by a bus someone else knows what’s going on.”
“So how do you keep the documentation in sync with what’s really going on?”
“You have to do that.”

Documentation gets you a document. It doesn’t get you guarantees.

I’ve had this conversation a few times and it doesn’t matter who or where it is. It’s the same story. When someone is surprised, someone asks “why wasn’t this documented!?”. Documentation is good, I agree. But only in the general sense. If you watch DHH’s keynote talk on progress, overstating the general case is very similar. He says progress is good in the general sense. But when you start asking people to change, they get mad. They’ll say things like “why can’t we have it the way it used to work?”.

The same goes for documentation. Everyone agrees that it’s good. But if you ask for time to gather the content for it, we can’t do that right now. If you ask for a tool that might help you automate “documentation” then we can’t fit that in a schedule. How many times have people not documented or written an email because they don’t want a paper trail to bite them in the ass? What about closed door, stealth meetings and initiatives? Do industry secrets get written down? Does McDonalds’s secret sauce get documented?

If the documentation has content that someone doesn’t like, maybe we’ll skip that or edit it. How about this:

Section 5.41.29
Our system is finicky and buggy because no one wants to take risks and we couldn’t try to solve this problem in the middle of development. So you have to restart the service every 5 minutes. Here’s how you do that.

An important point! We could write down “restart this every 5 minutes” but wouldn’t someone want to know why we are restarting things all the time? What was the reasoning for this kludge? What’s the history?

Obviously the tone above is bad. Let’s talk about the tone. When you ask someone to write something down so that it’s saved for future humanity, what are you really expecting? What style or model did you originally imagine when the desire for a doc hit you:

• Do you expect a training manual?
• A thesis explaining the theory and problem space the solution lives in?
• Do you expect an encyclopedia full of facts and truths?
• An insider’s fast-track guide to understanding what’s going on?
• A slick sheet full of capabilities and bragging rights?
• A wikipedia full of references, links and related material?
• A history-filled story with very few facts but reads cover to cover easily?

Each of these things have a style and an audience. Did you think about these things before you just said documentation? Did you communicate that to anyone? Does anyone understand your vision? Are we writing our docs to an audience full of laymans or experts? Do we even know? Most projects I’ve been on write to a really insulting level. XYZ for Dummies. That’s not good! Dummy books are not good!

You probably already have documentation out there. What does a wiki not solve for you? What does the README say? What is on your FAQ page? Do you have a test suite? Do you have an API? Do you have a demo or help file? Do you have a development and project methodology and does it have paper output like status reports? Does anyone really care about the content of status reports or are they just checking “are people working?” in which case that’s solving HR and people resources instead of showing what’s going into “the product”.

Every place I’ve been at values documentation. And when people leave the project or the company, it’s still a huge deal. All the documentation in the world doesn’t save the team from the shock of hearing that a valued team member is moving on to greener pastures never to talk to us again. Schedules slip, morale takes a nose-dive and someone is left holding the poop bag. If you are lucky, there’s a back-fill plan. For years, people will talk about employees who used to “be here”. And no one really runs to the documentation to find out how, what and why they worked. You tell the employee who’s leaving, “for the next two weeks, train someone and write down what you know”. And even then, that plan doesn’t work 100%. Loss still happens.

So if documentation is so valuable then why doesn’t it work? Are worker-bees revolting against doing it? Is our documentation even done? How do we know when our docs are done? Are they close to done now?

Just stop with the doc thing. Write your normal stuff. Your README or user manual. Develop a business methodology, keep it simple. Use jargon words and defined concepts to keep it short. Have hands-on people document USEFUL THINGS IN USEFUL WAYS THAT THEY WILL LIKE AND USE. If you ask them to open MS Word and start documenting, you have failed. Documentation is not a document.

Here are some things that hands-on people might like:

• A .plan file, dev log or equivalent to remind them what they were doing on Friday when they come in on Monday. – They will like this because Mondays will hurt less.
• A Kanban board for status. They will like this because status reports won’t be the only way to communicate what we are doing.
• Executable User Stores as answering the question “how it’s supposed to work”. They will like this because you won’t have to write a seperate test plan and test steps. Also, it lets devs take risks because they KNOW when everything is working. Also, it’s executable requirements because it’s your stories. I’m talking about Cucumber/Gherkin.
• A project blog. They’ll like this because it’s outward facing beyond status reports.

Ultimately, documentation is not the problem. Even when suggesting things above, it’s the stodgy management that is forcing everyone to write docs. If the customer wants a waterfall doc product then they are going to get one. Maybe there’s no choice. But is there? Do they like huge and out of date deliverables? Does anyone really know for sure?

Documentation is not the problem, it’s the whole project, it’s the whole culture. They don’t want to change their deliverables but they have never done anything different. They need to know that the project is going to succeed when in reality waterfall is designed to keep people from taking responsibility for when it doesn’t work. “We passed that gate! We can’t be blamed! You signed off on it!” And that’s really the point. Documentation is the tell-tale sign of infinite bureaucracy. Work for a car dealership and write down a doc on how to ride a horse. They’ll think “I didn’t know we had horses!” but no one will fire you for documenting a bunch of horse crap.

Final deploy.rb file here:
gist.github.com/3410415

Let’s Play with Capistrano from Chris Dillon on Vimeo.

Then I upgraded ruby a few times and that was ok. But then I sudo’d to root and got:
# ~/.rvm does not exist (or something)

Here are some things I didn’t like:

• Gemsets are great except when you are managing different global gemsets on different boxes.
• Maybe you work on a project that maybe doesn’t use RVM and you wonder if they really want an .rvm-gemset file in their project root.
• I loved upgrading my default ruby using RVM until it failed and I had to recreate all gemsets and reinstall all gems.
• Gemset exporting I never really got.

Rbenv is better?

So I’ve switched to no gemsets and rbenv. I just use rbenv and ruby-build. Rbenv seems to runs a lot lighter and doesn’t complain on sudo.

RVM is better?

RVM beats rbenv in gemsets though. Rbenv gemsets don’t have global so you have to create a pre-install rails just to `rails new`. It’s a chicken before the egg thing. There’s no global gemset so rails doesn’t exist and you don’t have a directory to create the .rbenv-gemsets file in. So you have to create a tmp rails project with like the Rails 3.0 generator. And then you have to upgrade that project or something.

update:This might not be true. If you delete your .rbenv-gemsets file out of your home directory and make sure that each project has it’s own gems installed then `gem` will be global and you can install/remove gems as normal. So your global gemset is when `rbenv gemset active` returns nothing.

Upgrading ruby versions with rbenv does not import your gems. So you have to wait until you need them and install them with `bundle` or pre-cd into each of your projects to install them?

So that’s why I went with a third option.

No Gemsets at all!

Bundler works but with no gemsets, you end up with a lot of dupe versions that don’t hurt anything. You can bundle exec foo and the correct version of foo will be loaded. However, I do miss having clean gemsets. This list grows and grows without end.

> gem search rails
*** LOCAL GEMS ***
rails (3.2.8, 3.2.2, 3.0.16)


Having dupe versions is kind of annoying after a while, especially if you wanted to create a mobile rails dev env on a usb key. You don’t know what your dependencies are. Or do you? You can go until a project with Bundler and do:

> bundle list
Gems included by the bundle:
* abstract (1.0.0)
* actionmailer (3.0.16)
* actionpack (3.0.16)
* activemodel (3.0.16)
...


Then I guess I could copy all the gems somewhere. But it’s not automatic. With gemsets, each project was in a directory and I could just grab all the *.gem files.

tl;dr

So there’s pros and cons to each method. I have no problems without gemsets but I don’t have cleanliness. I just don’t know what the downside to having a messy system is. Honestly, a patch to `gem` where gem list would just colorize the list from bundle list would be awesome.

The Arduino Mini Powered on a Breadboard

I was curious so last night I got the Arduino Mini up and running. It was pretty tricky. I tried powering it over 9v. I was careful to read up on sparkfun forums and the like to see what it could handle. I found that it would boot (or at least the power light would come on) on as little as 3.3v off the 9v battery. Then I measured what was coming out of the 9v and it was only 7v (old battery). I read that it could handle up to 9v on the 9v input pin. So I had that working but how do you load code on it?

So I had ordered this FTDI cable from sparkfun. It’s a 5v version as my Mini is a 5v too. An alternative is a breakout board with a regular USB host cable. So I messed around with the 9v powering my breadboard with a barrel jack and a little rocker switch and left the 5v off USB disconnected. So I was just trying to use USB for TX/RX information to program the Arduino (as in load code, not burn the bootloader).

First, I had to load FTDI drivers for mac. On Windows, this would create a COM port. On mac, it just magically works and there’s no control panel or sys prefs applet. That’s a bit weird that there’s no interface to change settings but luckily it all worked. I had read about Windows users having to set “rts on close” option to true on older versions of the Arduino IDE but in the end, that was not my problem.

I tried to upload the Examples->Blink sketch. I was getting a few errors. Like:
"programmer not responding" and
"avrdude: stk500_getsync(): not in sync: resp=0x00"

I had seen this before when I choose the wrong board type but after flipping through all possible combinations, this wasn’t it. I eventually ran into a post buried deep in the web that mentioned hitting the reset button on the Arduino right before you load the sketch. So I did that (with some tricky timing involved) and the not in sync error no longer happened. So this was great because this was the first success.

Success is so much more useful for progress than failure just because you can start messing with stuff. And up to this point I was wondering all kinds of things like: is my board bad? is my cable bad? are my drivers stupid? So I was happy that things were working but I didn’t like having to hit reset manually. I never had to do this on the regular Arduino.

So I looked at someone’s wiring and realized that the USB FTDI cable I was using has an RTS on it that can go to a reset pin. So I plugged that in. But then it would complain that it couldn’t find the programmer on the board. Then I realized that they didn’t just hook it up straight, they had a 100uf capacitor on it. So it acts like a little trigger. If it’s on constantly the Arduino is constantly resetting so the code upload sort of times out. When the cap hooked up, I don’t have to reset it manually just to update code on it.

So then I uploaded the blink sketch again and it “just works”. There’s no built-in LED on pin 13 so I just hooked one up (with a resistor, don’t forget an led doesn’t resist) and it blinked on pin 13. Pretty cool. Also it seems to boot really quick. Like almost instantly. When you power it up, it just starts blinking.

Also I was powering it off a 9v which worked fine. Apparently the mini has a regulator. I double regulated it to 3.3v and it still booted although the light is dim. I think it is pretty tolerant but I also read that it’s not as “tough” as the regular one. The regular Arduino is pretty forgiving to electrical errors (within reason… don’t plug it into the wall).

How Do the GPIO Pins on a Raspberry PI work?

I found a Python and a Ruby project for the Raspberry PI GPIO pins.

Here’s the python version. If you had an LED on pin 12, this code (I assume) would turn it on (high).

import RPi.GPIO as GPIO
# to use Raspberry Pi board pin numbers
GPIO.setmode(GPIO.BOARD)
# set up GPIO output channel
GPIO.setup(12, GPIO.OUT)
# set RPi board pin 12 high
GPIO.output(12, GPIO.HIGH)

The ruby version called gpio by klappy on github is pretty similar. I looked at the ruby code and it looks like an abstraction around the raw devices.

BASE_PATH = '/sys/class/gpio/'

So that’s pretty neat. You get something really similar to the Arduino but could a higher level language and have networking and video built in. That for me was what I was wondering. I don’t exactly understand how serial over raw devices and pins work but I do understand system and virtual device nodes in /sys.

class Car
  get_in_accident: ->
    @honk_horn("panic")
    @airbag()
  
  airbag: ->
    console.log("imma deploying mah airbag!")
  
  # just showing here that arguments don't change anything
  honk_horn: (level) ->
    if (level == "pissed")
      console.log("HONK HONK")
    else if (level == "panic")
      @yell("aaaa!", 1000)
      console.log("HON...")
  
  # the real issue is callbacks that lose the original scope
  yell: (message, wait_time) ->
    setTimeout ->
      console.log(message)
      @honk_horn("pissed")
      console.log("You jerk!")
    , wait_time

car = new Car
car.get_in_accident()

What you see in this example is:

HON...
imma deploying mah airbag!
aaaa!

timers.js:103
if (!process.listeners('uncaughtException').length) throw e;
^
TypeError: Object # has no method 'honk_horn'
at Object._onTimeout (.:29:14)
at Timer.list.ontimeout (timers.js:101:19)


@honk_horn is supposed to refer to the instance variable when yell is called. It doesn’t know about it’s self? Well, there’s a good chapter about this (Chapter 6 of this book) on Binding and the Fat Arrow in coffeescript. It’s a simple fix to a confusing problem. The method that is called later (the callback method) needs to have a fat arrow => which generates a __bind function in javascript when the coffescript is compiled. This saves you a lot of time and effort. All you have to remember is to use the fat arrow when a callback is going to be called out of scope.

So just change the setTimeout line to have a => instead of ->:

yell: (message, wait_time) ->
    setTimeout =>

Now it works right. Also notice that the “aaaa!” yell is happening before the “HON…” log statement in both examples.

HON...
imma deploying mah airbag!
aaaa!
HONK HONK
You jerk!