RudyGems

How big is my MySQL database?

mysql> SELECT table_schema,
  sum(data_length) / 1024 / 1024 "data",
  sum(index_length) / 1024 / 1024 "index",
  sum( data_length + index_length ) / 1024 / 1024 "total"
FROM information_schema.TABLES
GROUP BY table_schema \G;

*************************** 1. row ***************************
table_schema: aardvarks_development
        data: 5297.32812500
       index: 4407.93750000
       total: 9705.26562500
*************************** 2. row ***************************
table_schema: aardvarks_test
        data: 0.09375000
       index: 0.25000000
       total: 0.34375000
*************************** 3. row ***************************
table_schema: badgers_development
        data: 13.59375000
       index: 16.60937500
       total: 30.2031250

as you can see, aardvarks_developpment is almost taking up 10gig

if we only care about “test” databases, we can add a condition on “table_schema”

mysql> SELECT table_schema,
  sum(data_length) / 1024 / 1024 "data",
  sum(index_length) / 1024 / 1024 "index",
  sum( data_length + index_length ) / 1024 / 1024 "total"
FROM information_schema.TABLES
WHERE table_schema like "%_test"
GROUP BY table_schema \G;

*************************** 1. row ***************************
table_schema: aardvarks_test
        data: 0.09375000
       index: 0.25000000
       total: 0.3437500

Boom!

(credit to Paul Butcher for the query)

Updating RubyGems on Ubuntu 9.04

I don’t know how many times I’ve had to do this.

But every time I install Ubuntu again, I get this;

matthew@Rudyness:~$ sudo gem update --system
ERROR: While executing gem ... (RuntimeError)
gem update --system is disabled on Debian. RubyGems can be updated using the official Debian repositories by aptitude or apt-get.

The simple fix;

matthew@Rudyness:~$ sudo gem install rubygems-update
Successfully installed rubygems-update-1.3.5
1 gem installed
Installing ri documentation for rubygems-update-1.3.5...
Installing RDoc documentation for rubygems-update-1.3.5...
matthew@Rudyness:~$ sudo /var/lib/gems/1.8/bin/update_rubygems 
Installing RubyGems 1.3.5
RubyGems 1.3.5 installed

RubyGems installed the following executables:
	/usr/bin/gem1.8

matthew@Rudyness:~$ gem --version
1.3.5

From now on you can “sudo gem update —system” as much as you want.

Sweetness.

Installing Ruby 1.9.1 on Ubuntu 9.04

Now, I’m upset that Jaunty Jackalope doesn’t ship with ruby 1.9.1 as default,
and the packaged version is only 1.9.0.

So we’re going to have to build it from source.

And because we’re all progressive people, we’re not even going to worry about suffixing the executables with `1.9` I can’t be bothered to type `gem1.9`

The dependencies are simple;

sudo apt-get install build-essential libssl-dev libreadline5 libreadline5-dev zlib1g zlib1g-dev

Then the usual process…

mkdir ~/src && cd ~/src
wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p0.tar.gz
tar -xvf ruby-1.9.1-p0.tar.gz
cd ruby-1.9.1-p0
./configure
make
make test
sudo make install

And to get rails working properly

sudo gem update --system
sudo gem install rails

# and to get sqlite3 working properly
sudo apt-get install sqlite3 libsqlite3-dev
sudo gem install sqlite3-ruby

Done.

I’ll add to this if I notice any problems

Ruby 1.9 becoming the standard?

There’s a lot of talk recently about pushing the Ruby Community onto Ruby 1.9.

Now that’s cool, the more people using 1.9 the quicker the issues will get ironed out

(people need to be spurred into fixing their libraries)

But there is one thing standing in the way of 1.9 being adopted by the whole community, Default Packages

Mac OSX comes with 1.8.6 as standard, Ubuntu 8.10 comes with 1.8.7.

Whitney Houston said “the children are our future”, and that’s exactly who we need to start using 1.9 right from the word go.

With Ubuntu 9.04 (Jaunty Jackalope) and OSX Snow Leopard on their way out soon, surely if they adopted 1.9 as the default, we’d be on the road to success.

However, maybe that’s not going to happen.

A quick look at the packages of Jaunty Jackalope and you’ll see

This package is a dependency package, which depends on Debian’s default Ruby version (currently 1.8.x).

So I guess this thing’s not going to get fixed soon.

Expire eTAGs when you redeploy

Rails 2.2’s eTAG functionality is great.

def show
  @article = Article.find(params[:id])
  if stale_record(@article)
    # do something expensive
  end
end

protected

  def stale_record?(record)
    fresh_when(:etag => [current_user, record], :last_modified => record.updated_at.utc)
    !request.fresh?(response)
  end

“304 NOT MODIFIED” all around!

But this determines freshness based on just the record.
If I deploy a new version of the “show” template eTAG-compliant browsers won’t refresh.

I could build my own way of handling this inside “stale_record?”
But Rails already has a method for dealing with this.

All cache keys are generated through the following code.

if ENV["RAILS_CACHE_ID"] || ENV["RAILS_APP_VERSION"]
  expanded_cache_key << "#{ENV["RAILS_CACHE_ID"] || ENV["RAILS_APP_VERSION"]}/"
end

So what does it mean to be the RAILS_APP_VERSION?
Well, I’m going to mis-use it, and say “it’s the current version of our app”

And Capistrano already gives this to us… it gives us /current/REVISION!!!

Perfect!

A quick “/config/initializers/capified_etags.rb” later

app_version_path = File.join(Rails.root, "REVISION")
if File.exist?(app_version_path)
  ENV["RAILS_APP_VERSION"] = File.open(app_version_path).read.strip
end

Nice.

Let me know if there are any other approaches out there.

setting up a simple git server

On the internet you’ll see people talking about Gitosis, or just giving over all of their git requirements to github.

Github is great, but if you just want to do a small personal project, with an off-site repository, go set up your VPS as your main git server.

Here’s how you do it.

1. create a “git” user

   sudo adduser git

2. add your ssh keys to that user

   ssh-copy-id git@my.com

3. create a “bare” repository on your server

   mkdir myRepo.git
   git --bare init

4. add this as a remote source

   git remote add origin git@my.com:myRepo.git
   git push origin master

BOOM! that is literally all it takes!

Thanks to CommonThread for his tutorial on doing this.

Rails + Rack Middleware

Rails just released Metal!

Giving us a great framework to build custom Rack processors alongside our application.

Check out this great article over at Soylent Foo.

Recently we had a problem where Microsoft Office was somehow issuing OPTIONS request to our site.

This was blowing up our RESTful controllers that weren’t invited to the OPTIONS party.

We played with Apache rewrites based on REQUEST_METHOD,

but ultimately, we wasted a lot of time fighting against Rails and routing exceptions, blahblahblah.

But now Rails is running off Rack we can deal with it the correct way.

class OptionsCatcher < Rails::Rack::Metal
  def call(env)
    if env["REQUEST_METHOD"] =~ /OPTIONS/i
      [403, {"Content-Type" => "text/html"}, "OPTIONS requests are forbidden"]
    else
      super
    end
  end
end

B00M!

Now this is technically just middleware, and should be declared with a

config.middleware.use OptionsCatcher

and in lib/options_catcher.rb

class OptionsMiddleware
  def initialize(app)
    @app = app
  end
  
  def call(env)
    if env["REQUEST_METHOD"] =~ /OPTIONS/i
      [403, {"Content-Type" => "text/html"}, "OPTIONS requests are forbidden"]
    else
      @app.call(env)
    end
  end
end

but doing it as a Metal makes sense, and is easier than remembering the structure of middleware each time

But there seems to be a problem at the moment with this.

Will try and get a proper Metal working shortly.

has_one_accessor

HasOneAccessor is a plugin for ActiveRecord that solves the problem of having associations that act like a attribute.

Perhaps only a few users have an openid, so it doesn’t make sense to add an :open_id_url column to your :users table.

Instead you create a new model OpenId

class OpenId < ActiveRecord::Base
  belong_to :user
end

class User < ActiveRecord::Base
  has_one :open_id

  def open_id_url
    self.open_id && self.open_id.url
  end

  def open_id_url=(value)
    if value.blank?
      self.open_id && self.open_id = nil
    else
      self.open_id || self.build_open_id
      self.open_id.url = value
    end
  end
end

Now, that’s not too bad. But maybe you didn’t have time to clean it up.

Maybe you deal with it in your controller.

def update
  if params[:open_id_url] && params[:open_id_url].present?
    @user.open_id || @user.build_open_id
    @user.open_id.url = params[:open_id_url]
  end
  ...
end

Now, that’s all fine.

But we all want to make these things easier.

We all want to cast as much code aside.

ConventionOverConfiguration and all that.

Enter has_one_accessor

class OpenId < ActiveRecord::Base
  belong_to :user
end

class User < ActiveRecord::Base
  has_one :open_id
  has_one_accessor :open_id, :url, :allow_blank => false
end

This takes away all the code,

This takes care of building a record if there wasn’t one, of only saving it when you save the User.

It even kills the association if the value is :blank? (if you pass in :allow_blank => false).

B00m, h34dsh0t?

SolrQuery - build your solr queries dynamically in Ruby

I’ve been using ActsAsSolr for about 6 months.

And at work I developed some code to make SOLR queries work a bit like the conditions in ActiveRecord.

Check out my library at Github. http://github.com/matthewrudy/solr_query

SolrQuery.build(:keyword => "Feather duster"
=> "feather duster"

SolrQuery.build(:keyword => "clean",
                :organisation => [organisation1, organisation2])
=> "clean AND organisation:(275 OR 6534)"

Wicked!

Check out the readme for more examples.

UID problems across a shared mount?

We just moved to new Ubuntu servers.

We use file-column (with some modifications) to deal with our file uploads.

deploy@jgp-web01:/var/www/oursite$ ls -l public/uploaded_file/file_name/000/000/051/585/ 
total 4
-rw-r--r-- 1 gerhard www-data 2670 2008-11-15 05:09 MatthewJacobsCV.txt

deploy@jgp-web02:/var/www/oursite$ ls -l public/uploaded_file/file_name/000/000/051/585/ 
total 4
-rw-r--r-- 1 deploy www-data 2670 2008-11-15 05:09 MatthewJacobsCV.txt

If you look carefully, you’ll see one is owned by “deploy” and the other by “gerhard”.

These are mounted by NFS, shared across the two computers.

So these are actually the same file, but with different permissions on each.

A quick use of the “id” command confirmed this;

deploy@jgp-web01:~$ id deploy
uid=1004(deploy) gid=33(www-data) groups=33(www-data),1005(aspire)

deploy@jgp-web02:~$ id deploy
uid=1002(deploy) gid=33(www-data) groups=33(www-data),1003(aspire)

The answer is;

root@jgp-web01:~# usermod -u 10000 deploy

root@jgp-web02:~# usermod -u 10000 deploy

boom!

Now this is resolved, but you’ll have to chown all the existing files, back to `deploy`, else they’ll still be attached to the old uid.