Listen to Dave Thomas

Watch this video -> please. Start at 45:50 and watch at least 10 minutes.

I tell a lot of people about this video. I don’t tell 10,000 people up on a stage on some world tour. But whenever I get into a good discussion with geeks, I reference this video and tell them to watch it. So I tell a lot of people in terms of percentage in conversations that I’m enjoying. So in other words, I really like the video personally, it struck a chord (or a brain-thorn) and I want other people to consider his point of view. Specifically, Dave Thomas says you should learn a new language every year. Invariably, I’ll tell people that Dave Thomas writes books and a few times people will dismiss his main idea because they think he’s selling books.

Dismissing Dave Thomas’ opinion that you should learn a new language every year just because he is an author is a crap position to take. First, FUD. Second, Dave Thomas doesn’t sell books about the languages he says to check out in the video. Thirdly, I trust Dave Thomas. This isn’t some fucking Larry Ellison power play.

So in terms of following his advice, I might pick up Scala. It’s different enough and anti-hipster-omg-embrace-the-java-enterprise enough that it will challenge me and/or be different than say continuing with Python (will is really Ruby like Spanish to Portugese) learning. I’ve picked up a Scala book but haven’t started it yet. I looked at the Lift docs and was completely deflated. Chad Fowler tweeted about the Play! framework (thanks for the reminder) and thank god this looks better.

If I get sick of a city, I might move. When I move, I’m looking for something new. Similarly, starting out with Scala is going to be a bit like jumping into a new town. Ok, I need to find the equivalents of the old things that I’m used to. But wait, I moved to a new town to do something new! If I bring my old habits with me, I’ll just be writing Ruby code in Scala. And that’s crap.

Marge: I’ve dug myself into a happy little rut here and I’m not about to
hoist myself out of it.
Homer: Just bring the rut with ya, Honey.
– “You Only Move Twice”

What any technology stack is going to have to do on our first date

I need a list of libraries at least searchable by category and popularity. Popular gems are likely to be used and therefore good. This is great to get to higher level operations and gain productivity. For example, ruby-toolbox.com.

I need screencasts ordered by date so you can learn community trends. I’m going to need a way to keep up with the Jones’ and know what’s falling out of favor in the community because a shiny new library, technique or concept is getting the attention of key people. This also operates as curated content. For example, everyone is talking about CORS. I’m sure there will be a Railscast on CORS even though CORS is not specific to Ruby or Rails.

Related to that last point, the community should be of high quality. I know people like to hate on PHP but if you took a core sample of PHP-land, I think you wouldn’t be too happy. Of course there are exceptions, for example, Facebook is full of brilliance.

Some easy way to install things that’s Internet-enabled. Maven is close. Rubygems is good. NPM is good. Yum beats plain RPM. Pip is fine. CPAN is crap. All these package managers connect to the Internet and resolve dependencies. As a bonus, being able to repackage, bundle, freeze or tar up libs into a lib directory is great for corporate firewalls or making deployment not suck so hard.

I need a way to separate projects from one another. Gemsets with RVM does this. Virtualenv does this with projects. Lib implicitly does this in Java (although CLASSPATH issues suck). Explicit project isolation with libs is great.

Give me videos from the community like Confreaks. Video community talks are necessary. The JavaOne video site sucks and only because of the content. It doesn’t have to be flashy and overly-produced. Oracle publishes 3 minute videos with zero details and zero content. Yay?

I need quick feedback. This means a REPL and/or event-based test suite runners. If I save a file, I want a test to run (Ruby’s Guard gem). If I want to figure out how to do something, I want it to be instant feedback. About the farthest thing I can think of away from this is deploying to Tomcat, reloading the webapp and hitting refresh in a browser (#fml).

I’ll call Scala back

So far, Scala has the REPL thing down (just type $ scala). And the SBT tool does auto-reloading similar to Guard. Of course, it has to compile things so it’s slightly a slower feedback loop but it’s not terrible. SBT also seems to have a manifest thing going on for the equivalent of gemsets. It all goes in project/plugins.sbt which is great. There’s also a concept of a global project in a ~/.sbt folder. This reminds me a lot of NPM.

I haven’t discovered the community side of things yet like screencasts (railscasts) or a news site (rubyflow). On the negative side, there seems to be some Java crosstalk. For example the Play! framework can generate a Scala or a Java project. So if you are trying to get away from Java, this is a good step away but not a clean break. I get a feeling of Java refugee camp (which is fine).

Something I’m working on right now is understanding the IDE landscape. Eclipse was a bit weird with loading the Scala plugins and Intellij is coming out with a new version of IDEA soon. Also I need to grok the sbt-console inside the IDEA plugin vs command line. Are they the same? I had this same challenge when trying RubyMine. At some point you have to get a feel for what the One True Way was intended to be.

Great conference! There’s a lot to be said about it. I am not going to post it all, it would take a long of writing. You can find things about #coderetreat meetups out there.

After decompressing and reflecting, I came to a philosophical realization.

• Rubyists were not taught Ruby in school or elsewhere, they willed their skill into existence.
• Internet access came through the air over cell and we suddenly had dev environments in the middle of the woods.
• The wikipedia description of game of life was just some english bullet points that were implemented 100 times

So in these three ways, we willed things out of the ether and that is awesome.

For a less philosophical run-down of what happened, check out Brandon Hays’s blog.

I like to put a little date banner on files that are like a CHANGELOG. The Insert Comment Banner in Textmate is great for this. You just hit Ctrl+Shift+B and this appears:
=========
= Banner
=========
And the equal signs on top and bottom adjust as you type in “Banner”. It’s pretty cool. But what I wanted is a timestamp in the banner. So I’d leave Banner highlighted and then right click in Textmate and select “Filter through command” and put the Unix command date in there. That’s great except it puts Fri Jan 20 21:46:13 EDT 2034 in there which is a bit long.

===============================
= Fri Jan 1 21:46:13 EDT 2034
===============================

Too long. So here’s what you can do. Go to Bundles->Bundle Editor->Show Bundle Editor. Then go to the Source category. You can filter at the top for Snippets if it’s easier. See this picture.

Then put this code in:

This will give you a banner like this:
===================
= Fri Jan 1 2034 =
===================

And it’ll be the right length automatically. Nice.

Caching in Rails is really simple (straight from the docs):

You can set up your application’s default cache store by calling config.cache_store= in the Application definition inside your config/application.rb file or in an Application.configure block in an environment specific configuration file (i.e. config/environments/*.rb). The first argument will be the cache store to use and the rest of the argument will be passed as arguments to the cache store constructor.

Originally, I didn’t even know it was there. So I was thinking I’d just create a hash in memory. So I wrote my own cache class like this (homegrown_cache.rb):

Then you might want to use it in an implementation like like CacheEnabledSearch or something like that within your app.

But it’s way easier and smarter to just use the goddamn caching that’s built into Rails. Specifically, if you use the in memory store, since Rails 3.0 it’s thread safe. Meaning if two clients hit your cache, you’re not going to have weird bugs pop up. It’s very easy to configure and use.

I’m posting the Cache class anyway though it’s interesting how simple an object it is without being thread-safe. But even then, I’d recommend using the cache class from Rails outside of Rails.

So here’s how you’d use Rails caching “without” Rails. In other words, a console app.

Not particularly interesting. This would be way more useful inside of a Rails app where a singleton class called Rails.cache is set if you have something set in your config:
config.cache_store = :memory_store

Let’s avoid the Rails situation for now and create a file cache directory for a file based cache:
$ mkdir /tmp/poro_cache

Here’s a complete example.

If we run this one time, we get this:
ruby rails_cache_poro.rb
wrote to cache, run me again

And running it again:
ruby rails_cache_poro.rb
Reading from filestore cache: bob

Reset it with rm -rf /tmp/poro_cache

Ok, so that’s ActiveSupport caching outside of Rails. Wiring Rails up to do this inside a controller, lib class, helper or other ruby object is trivial once you have followed the caching guide. You won’t instantiate ActiveSupport::Cache anymore directly but you’ll do it indirectly through the config block for your environment.

And then all the statements will look like this:

But it works the same. Really easy. Of course, the next subject which is more complicated is when to invalidate your cache.

A common metaphor for a computer is that it’s a giant calculator. It’s easy to imagine a larger and larger machine. Pictures of a small desk calculator evolving into a slightly larger version, morphing into something the size of a breadbox with more buttons, changing into something the size of a refrigerator and finally we nod our heads and say “it’s a calculator but bigger”. That’s why it’s so powerful! It’s a huge calculator! But it’s not.

It’s a state machine.

What huge numbers would a calculator be unable to calculate anyway? You know that scientific notation
1.274921411340194917490141234 × 10^27? I’ve seen that in a calculator. Doesn’t that let the calculator handle a number of extremely large size? Is that why it can’t play games? Does a calculator not calculate enough? Or, is a state machine that is too slow to render real-time graphics.

It’s not that a calculator can’t store big numbers. It’s that a computer is so quick on changing its internal state that it can trick us with simulations and graphics.

A CPU is not a brain either but that’s a different post.

I encourage you to checkout the State Machine gem to play around with this concept. It has direct relevance to game development and other areas.

“We need to write down how it works, what we are doing and all the little tricks so that if you get hit by a bus someone else knows what’s going on.”
“So how do you keep the documentation in sync with what’s really going on?”
“You have to do that.”

Documentation gets you a document. It doesn’t get you guarantees.

I’ve had this conversation a few times and it doesn’t matter who or where it is. It’s the same story. When someone is surprised, someone asks “why wasn’t this documented!?”. Documentation is good, I agree. But only in the general sense. If you watch DHH’s keynote talk on progress, overstating the general case is very similar. He says progress is good in the general sense. But when you start asking people to change, they get mad. They’ll say things like “why can’t we have it the way it used to work?”.

The same goes for documentation. Everyone agrees that it’s good. But if you ask for time to gather the content for it, we can’t do that right now. If you ask for a tool that might help you automate “documentation” then we can’t fit that in a schedule. How many times have people not documented or written an email because they don’t want a paper trail to bite them in the ass? What about closed door, stealth meetings and initiatives? Do industry secrets get written down? Does McDonalds’s secret sauce get documented?

If the documentation has content that someone doesn’t like, maybe we’ll skip that or edit it. How about this:

Section 5.41.29
Our system is finicky and buggy because no one wants to take risks and we couldn’t try to solve this problem in the middle of development. So you have to restart the service every 5 minutes. Here’s how you do that.

An important point! We could write down “restart this every 5 minutes” but wouldn’t someone want to know why we are restarting things all the time? What was the reasoning for this kludge? What’s the history?

Obviously the tone above is bad. Let’s talk about the tone. When you ask someone to write something down so that it’s saved for future humanity, what are you really expecting? What style or model did you originally imagine when the desire for a doc hit you:

• Do you expect a training manual?
• A thesis explaining the theory and problem space the solution lives in?
• Do you expect an encyclopedia full of facts and truths?
• An insider’s fast-track guide to understanding what’s going on?
• A slick sheet full of capabilities and bragging rights?
• A wikipedia full of references, links and related material?
• A history-filled story with very few facts but reads cover to cover easily?

Each of these things have a style and an audience. Did you think about these things before you just said documentation? Did you communicate that to anyone? Does anyone understand your vision? Are we writing our docs to an audience full of laymans or experts? Do we even know? Most projects I’ve been on write to a really insulting level. XYZ for Dummies. That’s not good! Dummy books are not good!

You probably already have documentation out there. What does a wiki not solve for you? What does the README say? What is on your FAQ page? Do you have a test suite? Do you have an API? Do you have a demo or help file? Do you have a development and project methodology and does it have paper output like status reports? Does anyone really care about the content of status reports or are they just checking “are people working?” in which case that’s solving HR and people resources instead of showing what’s going into “the product”.

Every place I’ve been at values documentation. And when people leave the project or the company, it’s still a huge deal. All the documentation in the world doesn’t save the team from the shock of hearing that a valued team member is moving on to greener pastures never to talk to us again. Schedules slip, morale takes a nose-dive and someone is left holding the poop bag. If you are lucky, there’s a back-fill plan. For years, people will talk about employees who used to “be here”. And no one really runs to the documentation to find out how, what and why they worked. You tell the employee who’s leaving, “for the next two weeks, train someone and write down what you know”. And even then, that plan doesn’t work 100%. Loss still happens.

So if documentation is so valuable then why doesn’t it work? Are worker-bees revolting against doing it? Is our documentation even done? How do we know when our docs are done? Are they close to done now?

Just stop with the doc thing. Write your normal stuff. Your README or user manual. Develop a business methodology, keep it simple. Use jargon words and defined concepts to keep it short. Have hands-on people document USEFUL THINGS IN USEFUL WAYS THAT THEY WILL LIKE AND USE. If you ask them to open MS Word and start documenting, you have failed. Documentation is not a document.

Here are some things that hands-on people might like:

• A .plan file, dev log or equivalent to remind them what they were doing on Friday when they come in on Monday. – They will like this because Mondays will hurt less.
• A Kanban board for status. They will like this because status reports won’t be the only way to communicate what we are doing.
• Executable User Stores as answering the question “how it’s supposed to work”. They will like this because you won’t have to write a seperate test plan and test steps. Also, it lets devs take risks because they KNOW when everything is working. Also, it’s executable requirements because it’s your stories. I’m talking about Cucumber/Gherkin.
• A project blog. They’ll like this because it’s outward facing beyond status reports.

Ultimately, documentation is not the problem. Even when suggesting things above, it’s the stodgy management that is forcing everyone to write docs. If the customer wants a waterfall doc product then they are going to get one. Maybe there’s no choice. But is there? Do they like huge and out of date deliverables? Does anyone really know for sure?

Documentation is not the problem, it’s the whole project, it’s the whole culture. They don’t want to change their deliverables but they have never done anything different. They need to know that the project is going to succeed when in reality waterfall is designed to keep people from taking responsibility for when it doesn’t work. “We passed that gate! We can’t be blamed! You signed off on it!” And that’s really the point. Documentation is the tell-tale sign of infinite bureaucracy. Work for a car dealership and write down a doc on how to ride a horse. They’ll think “I didn’t know we had horses!” but no one will fire you for documenting a bunch of horse crap.

I can’t buy a decent router. This has been a decade long struggle and it’s about time I wrote about it. For a while I was doing network administration and Cisco stuff. I got my CCNA certification and learned to build Linux routers. I was knee-deep in firewalls, NAT and beginner network stuff. It eventually lost my interest but that’s another story. At this same time, I was running a Linux router at home for my roommates and myself. This is pre-WiFi so we ran 100mb ethernet all over the house and down the hallways. It worked fine until a hard drive (IBM Deathstar) died in the Linux router. I went through a series of hard drives in this box and it was kind of stupid because the OS really just needs to load up once and have some swap space (maybe). So although the Linux router was free (with an old PC) and stable, the reliance on a spinning disk was a problem.

Years passed and I had switched to the consumer router/wifi/switch combo box. It was the blue Linksys (pre-Cisco buyout) model. At that time everyone had one; it was super popular. My blue Linksys ran for a while but soon died mysteriously. I bought the Linksys GL one that runs some flavor of Linux so I could load custom firmware on it. This died so I switched to Dlink. This one started freezing up. At this point I was really frustrated because I had switched brands and strategies (custom firmware) and even moved from a Linux PC router to appliances (consumer routers).

You have to understand how black box these problems were. It’s not like they would freeze and flash some light, spit out an error or dump some log message. They just stop working. Suddenly you can’t ping them, or route out or my external IP is down. And it’s not ever related directly to anything. It’s not like I could run 100 copies of a program and bring it down. It’s always really random and sometimes would happen under light load. So when this would happen, roommates or spouses would get mad. My self-hosted stuff would be down. My Counter-Strike:Source server would be offline, a LAN party would be interrupted. This is over many years (10) and something (productive or fun) would always be impacted. So this last round of trying to buy my way out of the problem, I bought the most expensive router I had ever bought, a Buffalo WZR-HP-G300NH.

The Buffalo came with a vendor-branded version of dd-wrt. I had been running dd-wrt on my dlink for a long time and although I enjoyed the interface and features, it didn’t have any magic effect on this lockup issue. So I hoped that somehow the Buffalo would have more memory and/or be magically more compatible with dd-wrt. It wasn’t magic. It had the same lockup problems. The firmware that it ships with is based on dd-wrt but it is very old. It had Buffalo branding/logos on the admin page, I guess Buffalo just modifies or maintains the dd-wrt build. So I updated it to an official dd-wrt release. This worked for a while but then I had the lockup problems again. The problem is very random and intermittant so at this point when I say “it worked for a while”, this is inaccurate recall.

Even the dd-wrt wiki has a special page called Router Slowdown where they describe memory problems and too many open connections. No fault of their software, I would happily load it on an appliance with more memory. Unfortunately, building something small and quiet that perhaps fits in a hand-built 1U 19″ rack is a bit more expensive than just buying one from someone.

So recently I have bought a 1U small office router from Hacom. It has way more memory and hopefully has better quality memory so that it’ll crash less, if at all. I think if I had added up all the money I’ve spent on the consumer routers, it would add up to the Hacom. Eh, hindsight is 20/20.

It runs pfsense and I love it. It was really easy to set up even though the UI is pretty different. If you’ve been through a bunch of different routers, another UI is no big deal. It probably took me about 30 minutes to set up. When I ordered it, I got a personal call from Hacom and they answered a few questions I had! How awesome is that?! I asked about the fan noise since their newer models aren’t fanless and he said it was whisper quiet. It is but it has a tone. I can’t hear it around an open door corner so it’s pretty quiet but I wouldn’t put it in the same room as you work.

I’ve had no lockup problems and it tells me how much memory I’m using. I haven’t even hit 10% on memory use yet. I will update if any stability problems pop up.

I love my Drobo (disk pack). It was revolutionary at the time and it’s been working great for years. I took that love and bought a DroboFS a few years ago and fell in love again. It was a huge step up from my D-Link 323 which lasted about a year before I threw it out in anger.

The DroboFS has been throwing problems in my face lately which culminated in my losing all my data on it. I deserve any data loss incurred for not having a backup but in my defense, it’s a huge volume. Very expensive to mirror on another NAS. The error being thrown is:
Drobo detected an internal problem. Please contact Drobo Support by clicking on the link below:

This was mostly my fault. I had lost my password to the DroboFS but still had ssh access. So I reset the password over ssh after I found the .conf file but apparently a few days later some process really didn’t like this and I could no longer admin the Drobo over Drobo Dashboard. All my data was still in tact but eventually it wouldn’t finish it’s boot sequence. At this point, support could no longer help me so I just wiped my whole device rather than waiting for a fix.

That’s the problem with this all-in-one. The data is on the same disks as the config. The firmware is probably burned on some memory somewhere but the conf files, the encryption, the non-RAID layout is all on the disk. So if you screw up your Drobo, you lose your data. If you reset the Drobo to factory settings, you lose your data.

I don’t think any of this data loss was related to the recent 1.2.1 firmware update that they did but I did have some compatibility problems. My disk pack is EoL but the DroboFS isn’t.

I had reset my Drobo which meant there were no droboapps on it. So I loaded up apache, droboadmin and the nfs server. I also enabled time machine on a share. Some combination of this made it crash on boot. So now I had to wipe the config again which meant another data loss. I couldn’t even copy off data this time.

Anyway, I even tried replacing my one 750gb disk with a 1TB disk but I got in a weird state where it thought I had removed disks too quickly. All the disks were blinking red and I had to reset and wipe again. I think it had something to do with wanting to reset with drives removed but also having the name remain the same. At one point, I had a single drive in the Drobo but putting that single drive in later made it freak out because the box name was the same. Maybe it thought that it had failed but I was actually wanting it to wipe the old drive. I never pulled a drive while it was blinking and always when it was green but I think there’s some delay and lag that you have to be careful off, especially when inserting 5 drives at a time. The whole thing is really tricky and error-prone in a failure situation (I feel the same of Linux software raid).

In total, I had wiped my DroboFS 5 times. Now I think I have everything stable but I’m not going to load the NFS server. That means I’m going to have to use SMB or something from my Linux boxes.

I still really like the Drobo products. I just think eventually I’m going to buy a Synology just to have two different vendor NAS’s. But it’s an expensive experiment ($800 empty) so this all kind of really sucks.

Tried filling out hours and hours of captchas while watching the newborn. It didn’t go so well. I got sore hands and didn’t get $100 of free stuff. I ended up ordering a new arduino r3 anyway and a stepper motor to hopefully build a quiet baby rocker. I built a few previous iterations of a baby rocker out of Lego technic pieces but the motor was a little bit too loud and I’ve scrapped the project.

Anyway, here’s a screenshot to remind me of the captcha fun.

Git doesn’t work behind a proxy with homebrew (the macports new hotness). Because git:// is blocked at my office. There’s a patch here. Unfortunately, the drop-in replacement didn’t work for me (it’s an old commit).

Instead I made the modifications myself. Be warned, that this file will probably only work for the version I’m using (0.7.1), in which case you’ll have to look at the SVN commit yourself. :(

cd /usr/local/Library/Homebrew
cp download_strategy.rb download_strategy.rb.orig
wget http://squarism.com/files/download_strategy_proxy_fix.rb -O download_strategy.rb
export HOMEBREW_GIT_VIA_HTTP=1
brew install [something]


Any brew installs that use git should work now.

But then there’s curl. Curl doesn’t quite use the same env that others do. So solve it like this:

export http_proxy=http://proxy:80
export ALL_PROXY=$http_proxy

Curl likes that ALL_PROXY env for some reason.

After that, I was able to get my favorite homebrew apps installing:
brew install couchdb irssi git mysql watch lua p7zip htop openssl node npm nmap netcat

Update: If the above still doesn’t work for you (I had problems with Git URLs), try this:
cd /usr/local/Library/Homebrew
vi download_strategy.rb


Find the line when %r[^git://] then GitDownloadStrategy and replace (or comment it out) it with this:
when %r[^git://]
url.gsub!(/^git\:\/\//, ‘http://’)
GitDownloadStrategy

Git should use the http:// method of downloading code and brew install should work. Just to be clear, the relevant part of download_strategy.rb looks like this:

Another thing to try is to edit ~/.curlrc to enable a SOCKS5 proxy if you have one.
socks5 = "yourserver:port"