Using a Redis as a Database

Development,Ruby — squarism @ 11:37 am

pixel-ribbon_cor

The Spike

I was spiking on Redis recently. I wanted to use the redis-objects gem to simulate a shopping cart app even though the README specifically says

Just use MySQL, k?

I wanted to see what would happen if I tried it anyway. So the README and examples for the redis-objects gem are great so I’m not going to rehash what’s there. However, I will say though that the example has you hardcode the id field to 1. That detail snuck up on me.

If you don’t set an ID then you can’t work with a redis-object instance. You get an exception: Redis::Objects::NilObjectId: Attempt to address redis-object :name on class User with nil id (unsaved record?)

It’s basically trying to tell you, “hey, save the record first or set an ID”. Well, honestly, I don’t want to set an id myself. This is where the meat of the README is. Redis-objects really fits organically in an existing ActiveRecord model. That means Rails. In this case though, I don’t want an entire Rails app. I can see the value though in a plain old Rails app. Just look at the examples if you want to see more.

Anyway, continuing on with the spiking, I tried to integrate the Supermodel gem with Redis-objects. That sort of worked. You just class User < Supermodel::Base and you can sort of get it to work. This is great because Supermodel gives you finders like User.find_by_email('bob@yahoo.com') to make it act like ActiveRecord but you can't use .create(email: 'bob@yahoo.com') to begin with because of the same errors as I mentioned above. Redis-objects really wants the record to have an ID already. Even using Supermodel's RandomID mixin didn't work. The initialize order and callback hooks don't really work (or at least I couldn't get them to work).

Finally, I tried combining just redis-objects and datamapper redis. That worked. And it's pretty nice. Check it out.

So using this is pretty easy.

When you look at Redis, the keys are already composited for you and magic has happened.

Yay!

The name field is from redis-objects and the create uses datamapper. This is a really odd pairing but I like the fact that I have no sql database in the mix but still have finders similar to an ORM. Something to keep in mind, datamapper's finders are a bit different than the Rails 3 ones (no .where method).

Benchmarking A Million Things

Ok fine. So maybe this works, maybe it doesn't. Maybe it's not the right idea. What about the good stuff? Like, how fast can we load a whole lot of names into MySQL versus Redis using the above code and techniques? Is it even relevant?

A gist of these test results is here.

A More Complete Example

If you know the ID and don't need something like an auto-incrementing column outside your code/control then you can greatly simplify the code above by getting rid of Datamapper. You can simply use redis-objects to fake an ORM. I had great success using it as long as you USE NATIVE REDIS TYPES. Listen to the redis-objects author, don't try to force the tool into the use case.

Ruby p385 benchmarks

Ruby — squarism @ 2:14 pm

pixel-ribbon_electric_fun

I was playing around with the falcon p385 patch to see if it’s any faster than some of the more recent MRI rubies.
TL;DR version: looks like p192 is faster than p385 of any type or tweak.

Here’s how to get a p385 Ruby version patched with funny falcon’s performance patches using RVM.

mkdir ~/.rvm/patches/ruby/1.9.3/p385
curl https://github.com/funny-falcon/ruby/compare/p385...p385_falcon.diff > \
$rvm_path/patches/ruby/1.9.3/p385/falcon.patch
rvm install 1.9.3-p385 -n perf --patch falcon

Then rvm use 1.9.3-p385-perf or set it as your global ruby.

Test Setup

The following benchmarks were run on an i7 server with a RAID5 array. The disk is slow (lack of large cache) but the benchmarks were run on the same box so it should compare apples-to-apples.

From here on out, here are the definitions for the Ruby versions.
p194 = 1.9.3p194 default
p385 = 1.9.3p385 default
falcon = 1.9.3p385 with the above falcon diff patch applied
gcc_tweak = the falcon patches with GCC compile flags tweaked.

So what are these GCC tweaks? Explicitly setting the CFLAGS for your machine’s CPU type and recompiling ruby with the Falcon patches applied.

Micro and Macro Benchmarks

I used the ruby-benchmark-suite to run these tests.

Here are some example results. I can’t list them all. There are over 100 benchmarks. These are results for the mean times in seconds.

test 1.9.3p194 1.9.3p385 falcon gcc_tweak
bm_sudoku.rb 1.379112226 1.598182153 1.495923579 1.526717563
bm_open_many_files.rb 0.175602996 0.197096826 0.197673286 0.194135045
… etc etc

Here’s the winner summary for mean times. This is the number of times the ruby version was the fastest for a particular benchmark.
1.9.3p194 – 64 wins
1.9.3p385 – 29 wins
1.9.3p385-falcon with GCC tweaks – 10 wins
1.9.3p385-falcon 5 wins

Boot time and IO

Timing rails boot time is a bit more important to me. If you want to know how to really save “rails boot time” see the DAS screencast on not loading Rails at all.

Even when using domain objects and lib tricks, it’s nice to have Rails and all I/O boot fast. The main thing that funny falcon’s patches do is speed up requires and I/O.

So let’s benchmark booting a Rails app.
$ time bundle exec rake environment

Ruby Version Seconds
p385 patched with falcon and GCC tweaks 2.481 total
p374 defaults 3.336 total
2.0.0-rc2 2.613 total

In conclusion, p194 looks faster on the macro and micro benchmarks but Falcon patches boot Rails faster.

Hash Choices

Ruby — squarism @ 11:26 pm

bttf

As I’ve previously talked about, Hashes of Hashes are weird to work with. In the previous post about Captain Planet, I showed how to select, filter and manipulate 2D hashes and arrays but ultimately concluded that a hash of hashes is both weird and unnecessary (most of the time).

If you can control the data, inline your key into the hash data and make an Array of Hashes. It’s really where it belongs. If you don’t, you’ll find yourself doing a few extra iterations or work. Below you’ll see a simple example of the two data structures.

In this case of an array of hashes, the data is easier to manipulate with array operations and filters. But before talking about matching on arrays of hashes, I want to talk about matching on dates.

So comparisons with dates work like you might expect. But fuzzy matches do not. Take this example. Is Marty in between Doc Brown and the distant future? The answer should be true.

We use a Range (x..y) object to make a date range. Then we can use === to check if we get a match.

But wait, === on Date is different than === on Range. The method === on Date checks to see if it’s the same Date whereas === on Range checks to see if the argument is within the range. So if you flip it, it returns true.

So what we did in the middle there was flatten the hash of hashes into an array of hashes by merging the key with the ‘data’ part of the hash. Hopefully that makes sense.

New Hawtnesses

Rails,Ruby — squarism @ 12:04 pm

Wow, how can anyone keep up?

Zeus

Video of Zeus in action. Is it the new spork?

Super fast Rails boot times, but not just booting. Everything! Like DRB on steroids.

FishFish shell

`brew install fishfish`. The new ZSH? A fork of the Fish shell. Featured in the above video link. Be sure to uninstall fish through homebrew if you have already. It’s not the same thing.

YADR


YADR – for me, the new codegram/vimfiles? Impressive documentation, will really change my muscle memory.

Prezto

Included with YADR, looks to be a replacement for oh-my-zsh (my fav). Will have to try it out and update this.

Ruby DCamp 2012!

Blog,Ruby — squarism @ 5:47 pm


Great conference! There’s a lot to be said about it. I am not going to post it all, it would take a long of writing. You can find things about #coderetreat meetups out there.

After decompressing and reflecting, I came to a philosophical realization.

  • Rubyists were not taught Ruby in school or elsewhere, they willed their skill into existence.
  • Internet access came through the air over cell and we suddenly had dev environments in the middle of the woods.
  • The wikipedia description of game of life was just some english bullet points that were implemented 100 times

So in these three ways, we willed things out of the ether and that is awesome.

For a less philosophical run-down of what happened, check out Brandon Hays’s blog.

Making Rails Development on Windows Not Suck

Ruby — squarism @ 6:04 pm


One of these strategies will probably work for you.

I have a Windows 7 box. Normally I dev on Mac (not perfect but sucks less). I had built a Linux VM (ubuntu) but wanted to share my projects from the VM to windows to use sublime text 2. Editing through the shell is ok but I wanted a full text editor instead of vi (which I haven’t gotten my speed up on). So I normally watch my files for changes with Guard as previously blogged.

So I share out my ~/projects directory through VMWare to Windows. Great! Now I can have all the unix tools in the VM and a GUI editor without doing the console screen painting through VMWare. The problem is, libevent doesn’t trigger through the shared folder feature of VMware. I have no idea why.

So I pivoted. I tried the windows rails installer. Even though it tries to pre-patch mingw32 right before the native gem install, it didn’t work well with native libs like nokogiri (just to name one).

So I tried cygwin and apt-cyg (apt-get for Cygwin, nice, sorta). It didn’t work well with compiling ruby even after googling flags and installing gcc. Was getting “could not find C compiler”.

So this journey continues. I installed the latest version of VMware workstation on Windows, going to try shared directory again with new vmware version and ubuntu 12.04. Maybe there will be some magic in the upgrades. I doubt it but I will try.

I could also try my inline rspec trick even though that still uses libevent for triggering the test execution.

If you have no idea what I’m talking about but am curious, leave a comment and I’ll make a video or follow-up.

Splitting a sentence in Ruby and keeping the punctuation

Ruby — squarism @ 9:51 pm

Update:Doh! You can just do text.split(/(\.)\s+/) to make the regex keep the delimiter with the parens group match. Then you have to join in groups and some other stuff. I keep the post below for posterity. :)

When you split a string on a character, you get an array back without the split character. When splitting sentences, the period goes away with the split. So we can employ some rather nasty behind regex tricks to remember the puntuation and still do the split. Pretty great!

Here’s some code I stole from stackoverflow that splits sentences fairly well. Pay no attention to the complexity of this unchained beast.

When you run it:

Hipsters are everywhere!
Even in Home Depot?
Flannel wrench set in ma face.
[Finished in 0.3s]

That’s great but it does a fat lot of good when trying to modify it to reverse sentences. Not to mention, regexes falling down on edge cases (the above doesn’t work for shouting sentences like this one?!!

Ok, whatever. That’s regex stuff. Let’s try applying what we’ve learned from our last post and create an inline rspec test to do what we want.

First, we’ll test the simplest case with See spot run. And then we’ll add multiple sentences in.

Works pretty well. We can keep adding edge case tests to handle weird punctuation situations or the dreaded “Mr. Smith goes to college!” sentence.

A Different REPL Workflow in Ruby

Ruby — squarism @ 8:42 pm

Normally what I do is open an editor and start coding. In Textmate, I’ll hit Cmd+R to just run it as a big procedural file for something simple. If it gets too complicated, I’ll put it in a class. If something is confusing, I’ll break out to irb. The problem with this workflow is that you have to copy and paste a lot of code back and forth to get to “the breakpoint”.

In rails, I’d just put a binding.pry and magically, I’m at the breakpoint to play around at that point in time. No copying and pasting. Nice.

After watching a bunch of destroyallsoftware screencasts though, I like his one setup from episode #7 where he inlines rspec tests. Combine this with watchr and I might have a new favorite.

Let’s say I had the following hello world example.

When I run it in Textmate with Cmd+R, it doesn’t do anything. Because there’s no “main”. So then I have to add:

Great! But wait a minute, when I use it in another class, it’s going to print “Hello Variable!” out. So then I take it out? Then to test it again, I put it back in? This workflow although easy kind of sucks.

So let’s just inline some rspec. But before we even do that, we need rspec and watchr. We are going to use watchr to watch our files for changes. It’s best to install rev (Linux) or ruby-fsevent (Mac) for filesystem event firing.

gem install watchr rspec ruby-fsevent

Next, run this in a terminal and put it so where you can see it run while in your editor:
watchr -e 'watch("hello.*\.rb") {|m| system("clear && rspec -c #{m};") }'

Like this:

When you save the file (even without changes), it should say:
No examples found.

Finished in 0.00007 seconds
0 examples, 0 failures

Ok, now let’s inline some rspec tests to test our class.

Now we will get this nice rspec to the right in color immediately upon saving:

So how about irb? What if we want to learn/play in the code? Well now that we have a class and a shell instead of Textmate, we can use pry again.

And now when we save, we’ll be popped into a Pry shell.

So give this a shot and see if you like this as an alt workflow.

Let’s Play with Capistrano

Rails,Ruby — squarism @ 11:05 pm

In which I try to learn capistrano for the first time. Pushed a simple Rails 3 app to a mac dev box running rbenv, zsh, nginx and passenger. It’s LONG and edited only for length. So strap in and observe a completely blind learning session.

Final deploy.rb file here:
gist.github.com/3410415

Let’s Play with Capistrano from Chris Dillon on Vimeo.

Rbenv vs RVM

Ruby — squarism @ 6:38 pm


I used to use RVM with gemsets and loved it. Wayne’s work is amazing and way beyond any shell-fu I know. I was super impressed as I started and used it for at least a year on my main dev box.

Then I upgraded ruby a few times and that was ok. But then I sudo’d to root and got:
# ~/.rvm does not exist (or something)

Here are some things I didn’t like:

  • Gemsets are great except when you are managing different global gemsets on different boxes.
  • Maybe you work on a project that maybe doesn’t use RVM and you wonder if they really want an .rvm-gemset file in their project root.
  • I loved upgrading my default ruby using RVM until it failed and I had to recreate all gemsets and reinstall all gems.
  • Gemset exporting I never really got.

Rbenv is better?

So I’ve switched to no gemsets and rbenv. I just use rbenv and ruby-build. Rbenv seems to runs a lot lighter and doesn’t complain on sudo.

RVM is better?

RVM beats rbenv in gemsets though. Rbenv gemsets don’t have global so you have to create a pre-install rails just to `rails new`. It’s a chicken before the egg thing. There’s no global gemset so rails doesn’t exist and you don’t have a directory to create the .rbenv-gemsets file in. So you have to create a tmp rails project with like the Rails 3.0 generator. And then you have to upgrade that project or something.

update:This might not be true. If you delete your .rbenv-gemsets file out of your home directory and make sure that each project has it’s own gems installed then `gem` will be global and you can install/remove gems as normal. So your global gemset is when `rbenv gemset active` returns nothing.

Upgrading ruby versions with rbenv does not import your gems. So you have to wait until you need them and install them with `bundle` or pre-cd into each of your projects to install them?

So that’s why I went with a third option.

No Gemsets at all!

Bundler works but with no gemsets, you end up with a lot of dupe versions that don’t hurt anything. You can bundle exec foo and the correct version of foo will be loaded. However, I do miss having clean gemsets. This list grows and grows without end.

> gem search rails
*** LOCAL GEMS ***
rails (3.2.8, 3.2.2, 3.0.16)

Having dupe versions is kind of annoying after a while, especially if you wanted to create a mobile rails dev env on a usb key. You don’t know what your dependencies are. Or do you? You can go until a project with Bundler and do:

> bundle list
Gems included by the bundle:
* abstract (1.0.0)
* actionmailer (3.0.16)
* actionpack (3.0.16)
* activemodel (3.0.16)
...

Then I guess I could copy all the gems somewhere. But it’s not automatic. With gemsets, each project was in a directory and I could just grab all the *.gem files.

tl;dr

So there’s pros and cons to each method. I have no problems without gemsets but I don’t have cleanliness. I just don’t know what the downside to having a messy system is. Honestly, a patch to `gem` where gem list would just colorize the list from bundle list would be awesome.

Next Page »
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2013 SQUARISM | powered by WordPress with Barecity