A blind upgrade of all macports packages broke my ruby project because it uses imagemagick.


dlopen(/opt/local/lib/ruby/gems/1.8/gems/rmagick-2.13.1/lib/RMagick2.bundle, 9): Library not loaded: /opt/local/lib/libMagickCore.3.dylib
Referenced from: /opt/local/lib/ruby/gems/1.8/gems/rmagick-2.13.1/lib/RMagick2.bundle
Reason: image not found - /opt/local/lib/ruby/gems/1.8/gems/rmagick-2.13.1/lib/RMagick2.bundle


Massive hack incoming:

cd /opt/local/lib
sudo ln -s libMagickCore.4.dylib libMagickCore.3.dylib


There should be a better solution but I can’t find one until macports gets it’s goddamn *bEEp* straight. I swear to deity, every time I do a sudo port upgrade outdated, something breaks. I know it’s not macports fault but the nature of native ruby gem extensions but I’m just going to avoid upgrading.

Something else to consider is my recent switch to homebrew. I haven’t made the switch on all my boxes and the ImageMagick problem isn’t related to the package manager. I do prefer brew’s command structure. Moving MySQL wasn’t insanely difficult but I did cheat by just doing a stupid copy of the MySQL data files.

For a Linux development box, I typically install the same packages over and over again. For any new OS install, I call this the “One True Way” (OTW taken from a now defunct source online). Almost every box needs development libraries at some point. There’s a lot of extra stuff thrown in here too. These packages are circa ubuntu 10.04. Keep this in mind.

# aptitude install ruby ruby1.8-dev mysql-server mysql-client libmysql-ruby libmysqlclient-dev nmap htop nginx git-core postgresql nethack-console irssi tightvncserver xtightvncviewer distcc build-essential linux-kernel-headers linux-source couchdb solr-common solr-tomcat tomcat6 rar unrar p7zip k3b libk3b6-extracodecs gparted ubuntu-restricted-extras libdvdcss libxml2-dev libxslt-dev curl libcurl4-openssl-dev lua50 liblua50-dev libsqlite3-ruby libmagick9-dev

Download rubygems and install.

Add git hub to our gems list:# gem source -a http://gems.github.com.

Install some gems I like. # gem install mysql RedCloth capistrano rmagick termios warbler sinatra feedzirra

Install Chrome. Download .deb 64 package (google-chrome-beta_current_amd64.deb or newer) and double-click on the deb file. Click install.

Grab proggy fonts: clean, square, tiny, small. Set terminal font to proggy font. Set scrollback to 9000 lines, hidden scrollbar (shift+page up works instead), turn off menu, turn off terminal bell.

Install dropbox.

Go to town.

Git doesn’t work behind a proxy with homebrew (the macports new hotness). Because git:// is blocked at my office. There’s a patch here. Unfortunately, the drop-in replacement didn’t work for me (it’s an old commit).

Instead I made the modifications myself. Be warned, that this file will probably only work for the version I’m using (0.7.1), in which case you’ll have to look at the SVN commit yourself. :(

cd /usr/local/Library/Homebrew
cp download_strategy.rb download_strategy.rb.orig
wget http://squarism.com/files/download_strategy_proxy_fix.rb -O download_strategy.rb
export HOMEBREW_GIT_VIA_HTTP=1
brew install [something]


Any brew installs that use git should work now.

But then there’s curl. Curl doesn’t quite use the same env that others do. So solve it like this:

export http_proxy=http://proxy:80
export ALL_PROXY=$http_proxy

Curl likes that ALL_PROXY env for some reason.

After that, I was able to get my favorite homebrew apps installing:
brew install couchdb irssi git mysql watch lua p7zip htop openssl node npm nmap netcat

Update: If the above still doesn’t work for you (I had problems with Git URLs), try this:
cd /usr/local/Library/Homebrew
vi download_strategy.rb


Find the line when %r[^git://] then GitDownloadStrategy and replace (or comment it out) it with this:
when %r[^git://]
url.gsub!(/^git\:\/\//, ‘http://’)
GitDownloadStrategy

Git should use the http:// method of downloading code and brew install should work. Just to be clear, the relevant part of download_strategy.rb looks like this:

  when %r[^bzr://] then BazaarDownloadStrategy
  #when %r[^git://] then GitDownloadStrategy
  when %r[^git://]
        url.gsub!(/^git\:\/\//, 'http://')
        GitDownloadStrategy
  when %r[^hg://] then MercurialDownloadStrategy

Another thing to try is to edit ~/.curlrc to enable a SOCKS5 proxy if you have one.
socks5 = "yourserver:port"

For some reason I started researching server side websockets with a python module for apache. I thought, “oh this would be cool to create a reverse proxy or a socket to a port that’s not accessible through a firewall”. You could just have an apache module create a path back to some service.

Not knowing a lot about websockets, I kept reading and eventually realized that this problem would be best solved in an app server. The websocket module I found was written in python but if I used something like passenger to reverse back to a rails app server and then I could use whatever language I wanted. Besides, the web server shouldn’t be running code.

Anyway, fizzled idea.

I was reading a thread on stackoverflow about randomness and cryptography security and there was a comment that started me down a series of articles talking about the Monty Hall problem:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

The take away from the probability puzzle is that you should switch. It’s completely unintuitive and I didn’t really get it even after reading about it for some time until I watched this youtube video. It’s a bit slow paced but it’s the best explanation and visuals out there.

So even after grok’ing the concept, I wanted to prove it. So I wrote a ruby program to simulate the game show. Here’s the results from one run:
Monty Hall Games Played: 1000
Player 1 (switches): 675/1000 : 67.50%
Player 2 (stays) : 325/1000 : 32.50%

The code is pretty simple and quick. Of note is that I sort of use the same game and player_2 just picks the same door of the same game using the same rand(3) number. So this is like two people playing at the same time and one always switches. I used the sleep statement near the bottom to keep the terminal from blinking. Seems to not blink on Mac or Linux but I remember having some weird behavior somewhere.

#!/usr/bin/ruby
# a simulation of the monty hall problem
 
# monkey patch
class Array
  def shuffle!
    size.downto(1) { |n| push delete_at(rand(n)) }
    self
  end
end
 
class Player
  attr_accessor :name
  attr_accessor :choice
  attr_accessor :played
  attr_accessor :won
  attr_accessor :lost
 
  def initialize(name)
    self.name = name
    self.played = 0
    self.won = 0
    self.lost = 0
  end
 
  def pick_door(door_number)
    self.choice = door_number
    #puts "PLAYER PICKS: #{self.choice}"
  end
 
  def win
    self.won += 1
    self.played += 1
  end
 
  def lose
    self.lost += 1
    self.played += 1
  end
 
  def percentage
    num = (self.won.to_f/self.played.to_f) * 100
    sprintf('%.2f', num) + "%"
  end
 
end
 
class Game
  attr_accessor :doors
  attr_accessor :door_states
 
  def initialize
    # set up doors
    self.doors = Array.new
    self.doors = [ "goat", "goat", "car" ]
    self.doors.shuffle!
 
    # set up door states
    self.door_states = Array.new
    self.door_states = [ "closed", "closed", "closed" ]
  end
 
  # main loop
  def run(player_1, player_2)
    # Player 1 randomly picks a door
    # Player 2 uses this door for simplicity
    player_choice = rand(3)
    player_1.pick_door(player_choice)
    player_2.pick_door(player_choice)
 
    # Host picks a door with a goat and reveals door
    goats = []
    self.doors.each_index {|d| goats << d if doors[d] == "goat"}
    goats.delete player_choice
 
    # if player won already then there are two goat doors, host picks one
    if goats.length > 1
      goats.delete_at rand(2)
    end
 
    # first is redundant but safe
    door_states[goats.first] = "open"
 
    # Player 1 switches between two remaining doors
    # Player 2 does not switch so do nothing
    remaining = []
    self.door_states.each_index {|d| remaining << d if door_states[d] == "closed"}
    remaining.delete player_choice
    player_1.choice = remaining.first      # first is redundant but safe
 
 
    # Log result
    if win?(player_1)
      player_1.win
    else
      player_1.lose
    end
 
    if win?(player_2)
      player_2.win
    else
      player_2.lose
    end
 
  end
 
  # did player win?
  def win?(player)
    if self.doors[player.choice] == "car"
      true
    else
      false
    end
  end
 
  # print out the game statistics
  def stats(p1, p2)
    # terminal clear code
    print %x{clear}
 
    puts "Monty Hall Games Played: #{p1.played}"
    puts "#{p1.name} (switches): #{p1.won}/#{p1.played} : #{p1.percentage}"
    puts "#{p2.name} (stays)   : #{p2.won}/#{p2.played} : #{p2.percentage}"
  end
 
end
 
 
# Create players
player_1 = Player.new("Player 1")
player_2 = Player.new("Player 2")
 
# Create new game show
while player_1.played < 1000
  g = Game.new
  g.run(player_1, player_2)
  g.stats(player_1, player_2)
 
  # for terminal sanity if terminal blinks too much
  #sleep 0.02
end

It was a good exercise. I was playing around with ruby-ncurses but it became too much of a hassle. Maybe some other project will need a TUI, I do still want to play with ncurses or something similar.

I was working on my game store app and I had some TDD things I wanted to finish up before creating a rails3 branch. I discuss it kind of randomly in that post linked there. I had left it to languish because I got stuck and frustrated with cucumber. My app was working but the tests were failing (false negative). The biggest problem was trying to understand how to debug a cucumber step.

The first problem is logging. When cucumber runs webrat as a mock browser, I really expected to see the traffic in the server log (script/server). But it didn’t. Nothing. No traffic at all. The second problem was seeing what the browser sees. It was reporting back that my login was failing. The third problem is the problem itself. It was saying “invalid login”. I am using factorygirl to create mock users on the fly and authlogic for the authentication and so there was a possibility that something was wrong there.

Logging

So let’s go through this in order. First the log problem. The traffic I was not seeing in the script/server console was kind of confusing. Cucumber’s traffic is actually redirected to log/cucumber.log. So you can either tail -f that log or use the OSX console app (screenshot) to watch cucumber.log. Either way, the traffic and any puts or STDOUT text will be there. It’s pretty odd actually. I don’t know how they pull this off. The dev server (script/server) doesn’t even need to be running for the cucumber steps to work. See below, I stopped the server and ran the tests successfully:

box:game_store chris$ script/server
=> Booting WEBrick
=> Rails 2.3.8 application starting on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-12-05 00:21:34] INFO WEBrick 1.3.1
[2010-12-05 00:21:34] INFO ruby 1.8.7 (2010-08-16) [i686-darwin10.4.0]
[2010-12-05 00:21:34] INFO WEBrick::HTTPServer#start: pid=23554 port=3000
^C[2010-12-05 14:38:43] INFO going to shutdown ...
[2010-12-05 14:38:43] INFO WEBrick::HTTPServer#start done.
Exiting

2 scenarios (2 passed)
10 steps (10 passed)
0m0.508s


Debugging Webrat

Next, the “seeing what webrat sees” problem. The invalid login error was based on detecting text. In my test app, when you’re signed in you see the Logout link. When you’re signed out, you see the Login link. Webrat was telling me that it couldn’t sign in because Logout never appeared. I had to figure out if it was lying because the app in my dev environment was working fine. So I needed to see what webrat was seeing when launched from cucumber.

I had hunted down a function called save_and_open_page but it never worked for me. It’s supposed to take a snapshot of the webrat client and open the web page in your browser so you can see what it sees. It’s like debugging with puts. So if you’re stuck on a step, just save_and_open_page and investigate. Well unfortunately, I had left that option for dead a while ago because it wasn’t working. I had missed a key step:

Install the launchy gem.
gem install launchy

Now your default browser should launch with a simple snapshop of the page. It’s just a local file so you won’t be able to click on links back to your app (unless you have some awesome hardcoded urls everywhere that would take you back from anywhere). The idea here is to just look at what the client is seeing on a basic level. I was getting an invalid login error message so I was looking any evidence related to logins. You can see in the screenshot to the right that I have a Login link available. In my app, that means I’m not logged in. I put the save_and_open_page in the logout feature step of cucumber so this is a good thing.

Ok, so at this point my save_and_open_page was launching two pages. What? One was correct but the other one was incorrect. That is to say, one page was giving me an invalid login and the other one was signing in fine. So what is going on?

Factorygirl

A key to my problem was how I was using factorygirl to create mock users and misunderstanding a convention. First, here is the non-working features/step_definitions/user_session_steps.rb file:

require "authlogic/test_case"
require "factory_girl"
 
Factory.find_definitions
 
include Authlogic::TestCase
 
def user
  @user ||= Factory :user
end
 
def login
  user
  visit path_to("the homepage")
  click_link "Log in"
  fill_in "user_session_login", :with => "john" 
  fill_in "user_session_password", :with => "funkypass"
  click_button "Login"
end
 
 
def logout
  click_link "Logout"
  #save_and_open_page
  visit path_to("the homepage")
end
 
Given /^I am a registered user$/ do
  user
end
 
When /^I login$/ do
  login
end
 
Given /^I am logged in$/ do
  login
  visit path_to("the homepage")
end
 
When /^I logout$/ do
  logout
end

The login method is all about clicking the Log in link and filling in the username and password boxes. Now before you say, “what is john and funkypass?”. I tried many different variations of passwords. I tried my passwords out of my dev db, obviously this won’t work. I tried foo1/foobar which actually led to my one working and one not case.

So here’s what went wrong. My feature file defines two scenarios for one feature, logging in and logging out:

Feature: User Sessions
 
  So that I can blah, blah, blah
  As a registered user
  I want to log in and log out
 
  Scenario: log in
    Given I am a registered user
    And I am on the homepage
    When I login
    Then I should see "Login successful!"
    And I should see "Cart"
 
  Scenario: log out
    Given I am logged in
    And I am on the homepage
    When I logout
    Then I should see "Register"
    And I should see "Log In"

The “When I login” step fires the login method. But so does the “Given I am logged in” step. So there are my two save_and_open_page hits, the reason I see two pages open when debugging. But why would one work and the other one wouldn’t? I’m not changing anything.

The problem is factorygirl was creating a second user called foo2 for the log out scenario but rolling back the foo1 user that was created for the log in sceneario. I didn’t realize this when I hardcoded fill_in "user_session_login", :with => "foo1" during integration. So in the log out scenario, foo1 would log in but only foo2 would exist in the DB.

So I changed the login method to look like this:

def login
  @user ||= Factory :user
  visit path_to("the homepage")
 
  click_link "Log in"
 
  fill_in "user_session_login", :with => @user.login
  fill_in "user_session_password", :with => @user.password
 
  click_button "Login"
  #save_and_open_page
end

I really wanted to create a user once (what the ||= is trying to do) but I couldn’t figure out how to properly use instance variables in a test step. So I’ll just do it this way for now. Notice that I’m using the fields on @user so it’ll always work.

LibXML warning with Nokogiri

You may have noticed that I was receiving an warning here:
WARNING: Nokogiri was built against LibXML version 2.7.7, but has dynamically loaded 2.7.3
I’m on OSX and I’m not sure what OSX is using but xml2-config –version shows 2.7.3. So I’m pretty sure that it’s using the shipped OS version. Ok, no problem. I’m running homebrew (a macports/fink replacement). I’ll just install 2.7.7:

$ brew search xml
blahtexml libwbxml xml-tooling-c xmlrpc-c xmltoman
gccxml libxml2 xml2rfc xmlstarlet
html-xml-utils xml-security-c xmlformat xmlto
$ brew install libxml2
==> Downloading ftp://xmlsoft.org/libxml2/libxml2-2.7.7.tar.gz
######################################################################## 100.0%
==> ./configure --disable-dependency-tracking --prefix=/usr/local/Cellar/libxml2/2.7.7
==> make
==> make install
==> Caveats
...
$ brew info libxml2
libxml2 2.7.7

Ok so it installed correctly. Now we need to reinstall nokogiri using the 2.7.7 libs.

gem install nokogiri -- --with-xml2-lib=/usr/local/Cellar/libxml2/2.7.7/lib --with-xml2-include=/usr/local/Cellar/libxml2/2.7.7/include


Now when we run cucumber, we won’t get the error. But here’s the problem. Brew provides a way to link to the brew version of libxml2 by default with the `brew link libxml2` command but issues a warning about doing this. The problem with using a path like /usr/local/Cellar/libxml2/2.7.7/lib is eventually brew will update it to 2.7.8 or something and it’s hard to say what the consequences will be. Most likely, some binary will fail with lib errors.

Wrapping up

Another note is that I was futzing with all of this in a git branch called “tdd”, a branch I used to integrate cucumber/rspec and anything else related to testing. I read some advice somewhere that a good practice is to branch a new feature and then roll it back into master when it’s done.

So I was already in the “tdd” branch had already committed everything. It was time to merge the branches and get rid of the tdd branch:
# switch back to master
git checkout master
# merge tdd into master
git merge tdd
# run the tests again for sanity
cucumber features/user_sessions.feature
# ok, passed, we don't need the tdd branch anymore
git branch -d tdd
# view branches
git branch
* master


After the merge, my code is up to date in the master branch and I can create a new branch called “rails3″, for example, if I want to play with upgrading the app to rails 3. Or I could create a branch called refactor if I was going to majorly overhaul code.

I had watched Alicia Silverstone ‘hack’ a system recently and decided to make a mostly functional mock up like they do in the movies. I’m getting pretty comfortable in Processing and even though I really can’t stand Java anymore, I wanted to get this thing done and out of my head. I was just going to skip over this idea but inspiration just started flowing and I was idly solving it while getting coffee etc and I just had to get it on paper/code.

So here are the things it does:

• Switches between two GameState objects login and success
• Separate the view and the state (more on this)
• Has a super cool blinking cursor
• Handles shift, ctrl and meta (windows key) gracefully
• Handles backspace
• When you type muffin, it takes you to the super secret database … or whatever
• Goes fullscreen (which you can’t see in this shot)

This could get out of hand real quick. First, I reused a ton of code from Tatris. But in Tatris, the game state was tied to a view. I had pictured this project having different views. Like instead of a password prompt, maybe there’d be a keypad with a combo. Same state object (authorized vs unauthorized) so I didn’t need to recreate the wheel. The problem is, I don’t really have a controller so I’m kinda breaking MVC. But I didn’t want this thing to bloat up so I just went with it.

Next, UI layer. Not great. I’m drawing text boxes and blinking cursors. It’s really pretty messy. I’d be better off picking some UI components or something and using those. It’s tough. Even in games, there’s so many UI differences. Different scrollbars, different OK type buttons. What a usability nightmare. But I’m trying to create an old school terminal so all the weirdness is actually good.

Anyway, it was a fun little diversion. There’s more on the github page.

Recently, I’ve resigned from my position to pursue a more flexible and collegiate atmosphere at another company. I’m recklessly optimistic almost always but I feel like this is a good change for me. Not that there weren’t smart people at Oracle, it was just a little crazy at times and I needed free time cycles to move into more bleeding edge areas instead of picking up more and more enterprise software skills. Ok, enough about that. The important part is that I negotiated a month off at the new place for studying and ramping up on new technologies. Very awesome of them.

An interest of mine is NoSQL or alternative databases and scaling massive hashtables. There’s a possibility that I might be working with a dataset so large that it can’t be solved with an ETL style process. This is timely because I was just watching a video from Cloudera about Facebook having the same problems with large datasets on Oracle DB and then moving to Hive + Hadoop. Hopefully, I can hack on this to expand out my DB skills (as a dev) to a non-relational platform. There’s some guides on setting up a cheap cluster on EC2. Although I have some hardware, I think I’ll go that route. At the very least I’ll understand more about how Facebook scaled and how Mapreduce can be implemented. I hope to find a non-trivial problem to solve (ie: something beyond counting words).

In addition to that very new and complex goal, there’s some other things I want to hack on:

• MongoDB, especially as a horizontal scaling DB
• Rails3 upgrade and testing of new gems on my RSS analyzer project
• Cleaning up said RSS project to the point that I can share a 0.1 on github or what-not
• Continuing to hack on my Tatris port to Obj-C / iOS
• Break into node.js some more. Play with websockets.
• Amazon is giving away EC2 dev instances and I’ve never played with their weird command line tools and AMI thingies.

Of course there are distractions to all these very work relevant things. I have a languishing tetris clone that I haven’t touched since the summer and I never got around to adding sound effects to that. Producing the effects should be a problem but I’m not sure how hard it’s going to be to get back into that code. Thar be ghosts and cobwebs.

Game and graphics aren’t very work relevant (I haven’t found a simulation job). Even less so is playing around with my arduino. There are a few basics I want to get down and then possibly integrate them all into a functional thing. Something I would actually leave alone and serve some permanent purpose. I have a few ideas but I don’t want to foretell or foreblog. There’s also a RAD project I want to check out (Ruby on Arduino). I have no idea how it works or if it’s what I want to use.

Let’s say you’re trying to be nice to your wife/girlfriend/whoever and want to get the coffee in the morning before they wake up. So you set out to get a cup of coffee.

First, you can’t find your keys. Then you remembered that you loaned your house key to your neighbor Alice while you were out of town last week and you never got them back. You don’t want to wake up your special someone so you go to your neighbor Alice’s house and knock on the door. Alice says, “good morning!”
“Do you still have my house key?”, you say.
“It’s on my car’s keyring and my car is at my brother Barry’s house”, they reply.
“Where is your brother?”
“Oh, he’s across the street.”

So you walk across the street and try to find Barry. He’s pulling out of the driveway and he says “hop in!”.
“Well, actually I’m trying to get my car keys back”, you say.
“I’m in a rush, I need to deliver this couch to my friend Calvin who’s moving today. Would you mind helping me unload this and then I’ll give you your key back.”
“Sure Barry.”

So you unload the couch at Calvin’s house but when you’re putting the couch down, you rip a pillow. Calvin is none too happy.
“These pillows were stuffed with my finest Yak hair! You’re going to have to fix them”, says Calvin.
Barry says, “you get more stuffing and I’ll go get my needle and thread kit from my house”.

Barry leaves in his truck. Calvin gives you some yak shears and points you to his private Yak holding pen. Calvin goes inside and has some bacon and eggs. You sit down on a Yak shaving stool and start shaving a Yak on a Saturday morning. Suddenly you stand up and say, “why the frak am I shaving a Yak?!”.

“I just wanted some coffee!”

Yak Shaving is when you find yourself at the end of a long chain of implied tasks and find yourself suddenly shaving a Yak. You’ll find it a lot in technical tasks and projects but you’ll also find it in errands and daily life. Yak Shaving is wasted time or possibly some delusion of productivity. You should be having coffee in bed with your sweetheart but instead you’re loping off locks of Yak hair.

So let’s summarize our little tale and how it would end.

• You can’t find your keys because Alice has them.
• Alice sends you to Barry’s house.
• Barry takes you to Calvin’s house.
• You unload a couch and rip a Yak pillow.
• Calvin makes you shave a Yak.
• You finish shaving a Yak.
• Barry comes back and fixes the pillow.
• Barry takes you home and gives you your key back.
• You get coffee for your sweetheart.
• Your sweetheart can’t believe you shaved a Yak today.

Notice that Yak shaving happens at step #5 but there are 10 steps total. That’s because we’re in a chain. Just because you shave the Yak doesn’t mean you get your coffee. You still have to exit the chain which at the very least is N steps back out. Maybe on the way to get coffee, your car breaks (N+X). Who knows. So to avoid getting deep into a Yak chain, you need to break out of it as soon as possible. For example, tell Alice to get your keys as soon as possible and wake up your sweetheart and ask her where her keys are. Yes, it’s sub-optimal. But don’t assume that your sweetheart is going to be mad about being woken up. You could say, “honey I’m going out to get you a treat, where are your keys?”. It’s not that you should predict the future and assume that steps past Alice are going to be chaining, long and Yak-y. You should instead think of branching alternatives immediately and not just blindly follow Alice’s instructions.

Ok, so let’s forget about solving or avoiding the Yak problem. Let’s get into the meat of this. Let’s implement this chain in a few ways.

Ruby

# yak.rb
def get_coffee
        # getting coffee means two things, keys + car
        puts "I don't have my keys..."
        find_keys
        drive_to_coffee_shop
        puts "Got coffee."
end
 
def find_keys
        puts "Alice has my keys..."
        find_alice
        puts "Ok got keys."
end
 
def find_alice
        puts "Alice says Barry has my keys."
        find_barry
end
 
def find_barry
        puts "Barry wants me to help him unload a couch ..."
        unload_couch
        puts "Barry took me home and gave me my keys."
end
 
def unload_couch
        puts "I just ripped a Yak pillow unloading the couch."
        shave_yak
        puts "Done unloading couch."
end
 
def shave_yak
        puts "I'm shaving a Yak!"
end
 
def drive_to_coffee_shop
        puts "Driving to coffee shop..."
end
 
def surprise_sweetheart
        puts "Surprise honey!  Coffee!"
end
 
 
# Wake up, be nice to sweetheart
puts "I should surprise my sweetheart..."
get_coffee
surprise_sweetheart
puts "Awww..."

We run it and get:
$ ruby yak.rb
I should surprise my sweetheart...
I don't have my keys...
Alice has my keys...
Alice says Barry has my keys.
Barry wants me to help him unload a couch ...
I just ripped a Yak pillow unloading the couch.
I'm shaving a Yak!
Done unloading couch.
Barry took me home and gave me my keys.
Ok got keys.
Driving to coffee shop...
Got coffee.
Surprise honey! Coffee!
Awww...


Obviously this is very procedural and just creating the chain using simple method calls. But at least notice that the “main” of the program (the bottom five lines) just calls get_coffee and surprise_sweetheart. That part is at least somewhat encapsulated and clean. If we wanted to get really fancy we could create a generic action class that would receive messages and call a proc possibly. We’re still creating this chain in code which makes it extremely brittle and one-shot. How would we use this again? How would we put this in our toolbelt?

XML
Let’s create a XML-based workflow and run it in Ruby.

Here’s our steps from the bullets in an XML doc.

<?xml version="1.0" encoding="UTF-8"?>
<steps>
  <step stepId="0" stepName="Get Coffee">
    <message>I don't have my keys...</message>
    <step stepId="1" stepName="Find Keys">
      <message>Alice has my keys...</message>
      <step stepId="2" stepName="Find Alice">
        <message>Alice says Barry has my keys.</message>
        <step stepId="3" stepName="Find Barry">
          <message>Barry wants me to help him unload a couch ...</message>
          <step stepId="4" stepName="Unload Couch">
            <message>I just ripped a Yak pillow unloading the couch.</message>
            <step stepId="5" stepName="Shave Yak">
              <message>I'm shaving a Yak!</message>
            </step>
            <message>Done unloading couch.</message>
          </step>
          <message>Barry took me home and gave me my keys.</message>
        </step>
      </step>
      <message>Ok got keys.</message>
    </step>
    <step stepId="6" stepName="Drive to Coffee Shop">
      <message>Driving to coffee shop...</message>
    </step>
    <message>Got coffee.</message>
  </step>
  <step stepId="7" stepName="Surprise Sweetheart" onDone="Awww...">
    <message>Surprise honey!  Coffee!</message>
  </step>
</steps>

And here’s some code to step through the XML doc and print what’s going on.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

require 'rexml/document'
include REXML
 
xmlfile = File.new("yak.xml")
xmldoc = Document.new(xmlfile)
 
# Now get the root element
root = xmldoc.root
 
# hack
module REXML
  def each_child_element(&block)
    self.elements.each {|node|
      block.call(node)
    }
  end
end
 
def recurse(the_element)
  the_element.each_child_element do |childElement|
    if !childElement.attributes.empty?
      puts "[#{childElement.attributes['stepName']}]"
    else
      puts childElement.text
    end
 
    recurse(childElement)
  end
 
end  
 
recurse(xmldoc)

When we run it, we get this:

[Get Coffee]
I don't have my keys...
[Find Keys]
Alice has my keys...
[Find Alice]
Alice says Barry has my keys.
[Find Barry]
Barry wants me to help him unload a couch ...
[Unload Couch]
I just ripped a Yak pillow unloading the couch.
[Shave Yak]
I'm shaving a Yak!
Done unloading couch.
Barry took me home and gave me my keys.
Ok got keys.
[Drive to Coffee Shop]
Driving to coffee shop...
Got coffee.
[Surprise Sweetheart]
Surprise honey! Coffee!


It reads well but it’s not that great. First off, our XML file relies very heavily on the message elements being in the right order. I’d much rather have an onDone attribute on the element that would act like a callback. This is more in line with a workflow engine or something more dynamic that could execute code. I didn’t really have time to get this working. The message coming after the action is happening is supposed to signify what to do when it’s completed. This is not really that elegant. It’d be much better to create a queue and put messages onto it, then pop down the queue.

But that sounds like Yak Shaving to me …

This is a quick write-up without a lot of detail. We hacked together a quick rails app to do provisioning in the style of OIM behind an OAM SSO webgate. The complete guide and detail would be tens of pages so I’ll just give a quick overview for the strategy.

The Goal

• Ldap authentication
• No db
• Sso protected
• Weblogic deployment

Develop app steps and failures

We used activeldap for the LDAP pieces and defined our user model to narrowly search for a particular objectclass and attributes. Tried to use Authlogic. Fail. Acts as authenticated fail. Devise fail. Ended up using filters and activeldap. Integrating the gems and activeldap was actually kind of hard. A lot of the security gems assume you’ve got activerecord users and depend a lot on the validation helpers etc. So some of the authN gems didn’t work for us. We also had to hack a bit on the activeldap validations. Password policy was non-trivial. I just rolled my own like so:


def validate_password(password)
// initialize a password score
// call password rule methods like:
// check_special_characters
// check_length
// check_uppercase
end


The score from each check method is added to a total score. If the score is greater than zero then your password fails the checks. Each check, a hash for flash[:error] is used so that a precise error message is possible. It works ok except the flash error display is for some reason not ordered correctly.

All configuration constants are stored as YML as an app_config.yml file. For example, the LDAP server, port, password policy rules etc.

For the SSO config, we just detect header as HTTP_REMOTE_USER even though OAM is creating REMOTE_USER. Quick and easy. You have to append the “HTTP_” for the name. It’s a naming convention thing that you can’t do anything about. If your OAM header variable is UNICORNS, then you have to use HTTP_UNICORNS.

We used formtastic for the forms. This was a bit problematic with it trying to detect the activeldap model instead of activerecord.

Testing while developing

Ok so how do you test integration? Are you going to SSO enable your dev laptop? That’s way too hard. You can hardcode the credentials for a while but then eventually you’re going to want to test. I got around this by using a firefox plugin called modify headers. It’s pretty straight forward except for the small detail that you have to keep it open while hitting pages. I thought it would run in the background but it doesn’t. Just keep the modify headers firefox plugin open and it’ll let you create an auth cookie. Don’t worry, this isn’t a security hole. OAM in production won’t let you do this. It’s just used for development.

Warble

install warbler with gem install warbler
Generate default config warble config
Install jruby-openssl because activeldap requires it
Edit config/warble.rb to include jruby-openssl note that you don’t have to have jruby installed or anything.

The rest of the steps are not rails related. Deploy war to Weblogic as usual. Set up a Proxy webgate back to Weblogic for /app (you can’t protecte Weblogic directly with OAM). Protect /app with an OAM policy. If your firefox header test worked then when you turn that off and hit it behind OAM it will work the same. I was able to identify and trust the REMOTE_USER header coming in.

Bam, you’ve got a rails app working in a big scary enterprise SSO environment. The best part about all of this was how fast it went. Compared to JSP/Java EE dev, it was a breeze. The only big multi-day hangups we had was with activeldap. Many gems and auth models really expect you do have your user in the DB. Unfortunately, putting users in the DB creates a silo. Fine for small shops, not so good if you’re using Active Directory, OID, OpenLDAP or Fedora DS (389) for a centralized login.