tryruby.org
15 minute ruby intro. Recently bought by the code school guys.

codeschool.com
Has a free 2 hour rails tutorial called rails for zombies. This is really fun but longer. It’s totally interactive. Very well done. They have non-free classes too. Focuses on rails 3 and some advanced stuff.

railscasts.com
Not very hand-hold-y. I regularly watch this site just to learn new things and watch someone work. These are short in length and very focused.

rubykoans.com
Ruby Koans are practice exercises/meditations on different aspects of Ruby. There are many similar projects (Python Koans etc). It’s like an interactive cookbook. Very valuable to do even if you’re comfortable with Ruby. It’s a project that has failures built into it and it’s your job to fix them. I did a blog post on it. I’m not explaining this very well. It’s a bunch of tests set up to fail and you fix the first test case and it continues on. For example, you might set true = true and then the test suite will have 1 test passing and give you a progress bar. Very cool and easy to pick up and put down. Good for reference afterwards.

confreaks.net
Lots of conference videos organized by events. The HTML5 player sucks. Use the flash one, even that one is a bit picky with pausing and other stuff.

quickie_mart 20 min app
A plug! This is a talk that I did. The README is very short but has all the steps to build a store rails app. The repo is the completed app.

If you followed the instructions, you have the .txt files (which are CSVs) in your MySQL data directory (weird but ok). Importing CSV is really easy in MongoDB. Just make sure you are in the directory with the .txt files.


mongoimport -d geobytes -c cities --type csv --file cities.txt --headerline
mongoimport -d geobytes -c regions --type csv --file regions.txt --headerline
mongoimport -d geobytes -c dmas --type csv --file dmas.txt --headerline
mongoimport -d geobytes -c countries --type csv --file countries.txt --headerline


Now you can continue with the tutorial.

> db.cities.findOne();
{
"_id" : ObjectId("4fa2a734779f0ea93dd13df6"),
"CityId" : 42231,
"CountryID" : 1,
"RegionID" : 833,
"City" : "Herat",
"Latitude" : 34.333,
"Longitude" : 62.2,
"TimeZone" : "+04:30",
"DmaId" : 0,
"Code" : "HERA"
}


With a combination of spork, guard and rspec, none of my testing stuff was working. The spork prefork block was throwing this error.

Using RSpec
Preloading Rails environment
undefined method `gsub' for nil:NilClass (NoMethodError)


The solution was to comment all my test gems one at a time. And then I realized that my Ubuntu edited version of my Gemfile was conflicting with my Mac edited version of this same file.

So I made my Gemfile a little smarter.

 
group :test do
  gem 'rspec'
  gem 'rspec-rails'
  gem 'spork'
  # 
  gem 'rb-inotify' if RUBY_PLATFORM.downcase.include?("linux")
  gem 'rb-fsevent' if RUBY_PLATFORM.downcase.include?("darwin")
  gem 'guard'
  gem 'guard-rspec'
  gem 'guard-spork'
end

Then things started to work again. With any weird bugs in Rails, try commenting out gems one at a time.

The biggest change is how to pull new rubies in. What I mean is, let’s say a future version comes out. Let’s call it Ruby 14.0 so that this blog post doesn’t look dated for a really long time (#wat). If you wanted to update your global ruby with RVM, it’d go something like this: 1) rvm get latest 2) rvm install ruby-14.0 Then you’d migrate your gemsets or play with .rvmrc files.

Rbenv is a bit easier in this regard but it requires slightly more typing. I’ll also show you a little trick on how to incorporate some plugins neatly.


> rbenv versions
1.8.7-p357
* 1.9.3-p0 (set by /home/user/.rbenv/version)


Bah. We already dated this blog post. Anyway. 1.9.3-p125 is currently out. So let’s try to pull it in.


> rbenv install 1.9.3-p125
ruby-build: definition not found: 1.9.3-p125


“What? It’s out! I even just did this on another rbenv install! What is going on?”
[Keep raging, don't ship apps.]

So what’s happening here is the rbenv is so old that it doesn’t know what p125 is. So let’s update our rbenv install.


> cd ~/.rbenv
> git pull


Great. But that’s not the equivalent of ‘rvm get latest’ because we are using ruby-build to do the ‘rbenv install’. Now the default documentation has you checkout ruby-build in your home directory. So if you copied and pasted (like me) then you have a ~/ruby-build dir. Let’s move that to ~/.rbenv/plugins (make sure plugins is a directory). If you are using rbenv-gemsets to mimic rvm gemsets then you already have a plugin directory.


> ls ~/.rbenv/plugins
rbenv-gemset ruby-build

> cd ~/.rbenv/plugins/ruby-build
git pull


If you installed ruby-build into /usr/local then you can leave off the PREFIX variable.

> PREFIX=~/local ./install.sh


If you installed it into your home, you’ll need to modify your PATH variable:
# for ruby build and other PREFIX overridden builds

if [ -d "$HOME/local/bin" ] ; then
PATH="$HOME/local/bin:$PATH"
fi


If you want it in /usr/local, then leave off the above if statement etc and just run sudo ./install.sh like the ruby-build docs say.

Now rbenv knows what you’re talking about:

> rbenv install --definitions 2>&1 | grep p125


And now you can install the new ruby and set it to be your default. The good thing here is, unless you’re switching ruby versions, you don’t need to update all your gemset files like you need to in RVM. So for me, there’s less impact when keeping Ruby up-to-date.

def hello
  puts "Hi!"
end

Now let’s un-hardcode that string in the puts by adding a parameter.

def hello(greeting="Hi!")
  puts greeting
end
 
>> hello
Hi!
>> hello("Hola!")
Hola!

Great. We have a default String. But what about something more complex. What if we want a hash of options. Say we have a little piece of an IRC client.

def connect(options={})
  defaults = {
    :server => "irc.freenode.net"
  }
  options = defaults.merge(options)
  puts "Connecting to #{options[:server]} ..."
end
<pre lang="ruby">
 
Now when we use it like this, we can connect to a default server or override it.
<pre lang="ruby">
>> connect
Connecting to irc.freenode.net ...
 
>> connect({:server => "irc.efnet.net"})
Connecting to irc.efnet.net ...

Now a more complicated example. All we’re doing here is loading defaults from a YAML file and doing the same thing as before.

require 'yaml'
 
class Preferences
  def initialize
    if !File.exists?("preferences.yml")
      # example file
      options = {:server => "irc.efnet.net"}
      self.save!(options)
    end
    @values = YAML::load(File.open("preferences.yml"))
  end
 
  def to_hash
    @values
  end
 
  def save!(options)
    preferences = File.open("preferences.yml", "w") do |f|
      f.write(options.to_yaml)
    end
  end
end
 
def connect(options={})
  defaults = {
    :server => "irc.freenode.net",
    :username => "CHANGE-USER-NAME, see README.txt",
    :channel => "#chat"
  }
  options = defaults.merge(options)
  puts "Connecting to #{options[:server]} as #{options[:username]}..."
end
 
 
# Main
prefs = Preferences.new
 
options = {:channel => "#meow"}
connect(prefs.to_hash)
 
options[:username] = "Bob"
options[:server] = "irc.efnet.net"
prefs.save!(options)

Go ahead and give it a try and play with it. It’s a good recipe with many uses.

Years passed and I had switched to the consumer router/wifi/switch combo box. It was the blue Linksys (pre-Cisco buyout) model. At that time everyone had one; it was super popular. My blue Linksys ran for a while but soon died mysteriously. I bought the Linksys GL one that runs some flavor of Linux so I could load custom firmware on it. This died so I switched to Dlink. This one started freezing up. At this point I was really frustrated because I had switched brands and strategies (custom firmware) and even moved from a Linux PC router to appliances (consumer routers).

You have to understand how black box these problems were. It’s not like they would freeze and flash some light, spit out an error or dump some log message. They just stop working. Suddenly you can’t ping them, or route out or my external IP is down. And it’s not ever related directly to anything. It’s not like I could run 100 copies of a program and bring it down. It’s always really random and sometimes would happen under light load. So when this would happen, roommates or spouses would get mad. My self-hosted stuff would be down. My Counter-Strike:Source server would be offline, a LAN party would be interrupted. This is over many years (10) and something (productive or fun) would always be impacted. So this last round of trying to buy my way out of the problem, I bought the most expensive router I had ever bought, a Buffalo WZR-HP-G300NH.

The Buffalo came with a vendor-branded version of dd-wrt. I had been running dd-wrt on my dlink for a long time and although I enjoyed the interface and features, it didn’t have any magic effect on this lockup issue. So I hoped that somehow the Buffalo would have more memory and/or be magically more compatible with dd-wrt. It wasn’t magic. It had the same lockup problems. The firmware that it ships with is based on dd-wrt but it is very old. It had Buffalo branding/logos on the admin page, I guess Buffalo just modifies or maintains the dd-wrt build. So I updated it to an official dd-wrt release. This worked for a while but then I had the lockup problems again. The problem is very random and intermittant so at this point when I say “it worked for a while”, this is inaccurate recall.

Even the dd-wrt wiki has a special page called Router Slowdown where they describe memory problems and too many open connections. No fault of their software, I would happily load it on an appliance with more memory. Unfortunately, building something small and quiet that perhaps fits in a hand-built 1U 19″ rack is a bit more expensive than just buying one from someone.

So recently I have bought a 1U small office router from Hacom. It has way more memory and hopefully has better quality memory so that it’ll crash less, if at all. I think if I had added up all the money I’ve spent on the consumer routers, it would add up to the Hacom. Eh, hindsight is 20/20.

It runs pfsense and I love it. It was really easy to set up even though the UI is pretty different. If you’ve been through a bunch of different routers, another UI is no big deal. It probably took me about 30 minutes to set up. When I ordered it, I got a personal call from Hacom and they answered a few questions I had! How awesome is that?! I asked about the fan noise since their newer models aren’t fanless and he said it was whisper quiet. It is but it has a tone. I can’t hear it around an open door corner so it’s pretty quiet but I wouldn’t put it in the same room as you work.

I’ve had no lockup problems and it tells me how much memory I’m using. I haven’t even hit 10% on memory use yet. I will update if any stability problems pop up.

The DroboFS has been throwing problems in my face lately which culminated in my losing all my data on it. I deserve any data loss incurred for not having a backup but in my defense, it’s a huge volume. Very expensive to mirror on another NAS. The error being thrown is:
Drobo detected an internal problem. Please contact Drobo Support by clicking on the link below:

This was mostly my fault. I had lost my password to the DroboFS but still had ssh access. So I reset the password over ssh after I found the .conf file but apparently a few days later some process really didn’t like this and I could no longer admin the Drobo over Drobo Dashboard. All my data was still in tact but eventually it wouldn’t finish it’s boot sequence. At this point, support could no longer help me so I just wiped my whole device rather than waiting for a fix.

That’s the problem with this all-in-one. The data is on the same disks as the config. The firmware is probably burned on some memory somewhere but the conf files, the encryption, the non-RAID layout is all on the disk. So if you screw up your Drobo, you lose your data. If you reset the Drobo to factory settings, you lose your data.

I don’t think any of this data loss was related to the recent 1.2.1 firmware update that they did but I did have some compatibility problems. My disk pack is EoL but the DroboFS isn’t.

I had reset my Drobo which meant there were no droboapps on it. So I loaded up apache, droboadmin and the nfs server. I also enabled time machine on a share. Some combination of this made it crash on boot. So now I had to wipe the config again which meant another data loss. I couldn’t even copy off data this time.

Anyway, I even tried replacing my one 750gb disk with a 1TB disk but I got in a weird state where it thought I had removed disks too quickly. All the disks were blinking red and I had to reset and wipe again. I think it had something to do with wanting to reset with drives removed but also having the name remain the same. At one point, I had a single drive in the Drobo but putting that single drive in later made it freak out because the box name was the same. Maybe it thought that it had failed but I was actually wanting it to wipe the old drive. I never pulled a drive while it was blinking and always when it was green but I think there’s some delay and lag that you have to be careful off, especially when inserting 5 drives at a time. The whole thing is really tricky and error-prone in a failure situation (I feel the same of Linux software raid).

In total, I had wiped my DroboFS 5 times. Now I think I have everything stable but I’m not going to load the NFS server. That means I’m going to have to use SMB or something from my Linux boxes.

I still really like the Drobo products. I just think eventually I’m going to buy a Synology just to have two different vendor NAS’s. But it’s an expensive experiment ($800 empty) so this all kind of really sucks.

After a massive initial load of 155k gems, the size was about 45GB (currently, it grows pretty quick per week). The rubygem and gem mirror command is smart enough to just download just the deltas when you run it again:


$ gem mirror
Fetching: http://rubygems.org/specs.4.8.gz
Total gems: 170843
Fetching 16176 gems
................................................................


Then I wanted to know the size of all the latest gems only. If I had to do a lazy sneakernet, this might be one method of grabbing a whole bunch of dependencies (of course this would never work). Regardless of that, I still wanted to know what percentage of ruby gems space is old versions.

So I wrote a ruby program to find all the latest versions of the gem files and total up their size. I was not very happy about my experiments with #sort and #sort_by. The biggest problem is that it took 64 HOURS to run. I knew it had lots of problems but I didn’t want to kill it. I wanted to see how bad it really ran.

I’m not going to post the actual code. You can see the old version at this git commit url. The basic gist of the crappy algorithm was something like this:

Find all the files in the gem mirror off the filesystem.
Get the basename of the file name (ie: strip the path).  /tmp/foo-0.1.gem -> foo-0.1.gem
Go through all the basenames (gem names) find the gem family.

Here’s the problem. I had a massive list of 170k gems and then I’m trying to do a find_all right here to sort the gems into gem families. For example: there might be foo-0.1.gem, foo-0.2.gem and foo-async-0.1.gem. In this example, there are two gem families out of the three gems. Foo-async and foo are two different gems with their own versions. Later on, I would:

Do a version compare.
Push the latest version name to an array.
Delete the gem family name from the gem_names array.

Sounded good on paper. And then it took 65 hours to run (227305.19 seconds) and CPU was absolutely pegged the entire time. This algorithm was easy to come up with in IRB using a small test data set but scaling up in the real use case completely sucked. So I pushed it to github for versioning and rewrote the loop.

The latest version runs in 8.5 seconds and spits out a total size of all the latest ruby gems at 6.5GB. Of course, this information is useless since it’s not going to check compatibility or anything. I was just curious to know how much space is back versions.

The real key to the new version is the fact that I’m using a proper “grouped” data structure (Hash) instead of a massive flat Array. This allows the regexes and other operations to work on a smaller data set. The compound nature of the previous inefficiency is pretty amazing (hours to seconds).

So hopefully you see above that a huge array of a File glob is flat and makes regex’s or grouping operations very time consuming. Ruby’s magic group_by method sorts and groups the data structure once and then it’s much easier to regex out versions and do other things.

See below for the code inline or take a look at the github repo.

# refactored the 63 hour version to a much better 8.5 second version
 
require 'action_view'
include ActionView::Helpers
 
# change to location of rubygems mirror
GEM_DIR = "/opt/rubygems/gems"
 
gems = Dir.glob("#{GEM_DIR}/**/*.gem"); 1
gems = gems.collect {|g| g.split("/").last}; 
 
class Version
  include Comparable
  attr_reader :major, :feature_group, :feature, :bugfix, :version_string
 
  def initialize(version="")
    @version_string = version
    @major = "0"; @feature_group = "0"; @feature = "0"; @bugfix = "0"
 
    v = version.split(".")
    # puts v.join("|")
 
    if v[0]; @major = v[0]; else; raise "Major number blank."; end
    if v[1]; @feature_group = v[1]; end
    if v[2]; @feature = v[2]; end
    if v[3]; @bugfix = v[3]; end
  end
 
  # strangely enough .to_i works even for
  # >> "6-mswin32".to_i
  # => 6
  def <=>(other)
    return @major <=> other.major if ((@major.to_i <=> other.major.to_i) != 0)
    return @feature_group <=> other.feature_group if ((@feature_group.to_i <=> other.feature_group.to_i) != 0)
    return @feature <=> other.feature if ((@feature.to_i <=> other.feature.to_i) != 0)
    return @bugfix <=> other.bugfix if ((@bugfix.to_i <=> other.bugfix.to_i) != 0)
    # we probably have two things equal here
    return -1
    puts "FALLING THROUGH in <=>, not good"
  end
 
  def self.sort
    self.sort!{|a,b| a <=> b}
  end
 
  def to_s
    @version_string
  end
end
 
# temporary benchmarking
RubyProf.start
 
group_r = Regexp.new(/(.*)-(\d+\.\d+.*)\.gem$/)
gems_grouped = gems.group_by {|g| g.scan(group_r).flatten[0] }
# => {"firewool"=>["firewool-0.1.0.gem", "firewool-0.1.1.gem"}], ... }
 
latest_gems = []
 
gems_grouped.each do |g|
  versions = g[1].collect {|ver| ver.scan(group_r).flatten[1] }
  # => ["0.1.0", "0.1.1", "0.1.2"]
 
  begin
    latest = versions.collect {|v| Version.new(v)}.sort.reverse.first
    # => "0.1.2"
  rescue ArgumentError
    puts g
  rescue NoMethodError
    # somebody's got some crazy gem naming conventions
    # for example: chill-1.gem
    gems_grouped.delete g
  end
 
  latest_gems << "#{g[0]}-#{latest}.gem"
end
 
total = 0
latest_gems.each do |gem|
  begin
    total = total + File.size("#{GEM_DIR}/#{gem}")
  rescue Errno::ENOENT => e
    puts "WTF no #{gem}"
  end
 
end
 
puts "Total size of newest gems in #{GEM_DIR} is #{number_to_human_size(total)}"

Algorithm win. Rubygem mirror size curiosity complete. 6.5GB is current gems out of 45GB (right now).

Anyway, here’s a screenshot to remind me of the captcha fun.

A 20 minute Rails demo that I used as part of a “What is Ruby on Rails?” talk. The store was not designed or developed from scratch in 20 minutes but serves as a Cooking Show style demo of what is possible in a very short amount of time.

Quickie Mart on Github