A Trip Down Memory Lane With Derailed Benchmarks

This is the story of how I reduced the memory consumption of the phony_rails Ruby gem by 92%.

A few years ago, I decided to give Richard Schneeman’s derailed_benchmarks gem a try in a Rails project I was working on. The first aspect I profiled was memory used at require time:

$ bundle exec derailed bundle:mem

TOP: 120.1602 MiB
  phony_rails: 28.8711 MiB
    iso3166: 26.6094 MiB
    phony: 2.0195 MiB

The results above me told me that phony_rails was responsible for 24% of the app’s memory use, with the majority coming from iso3166. I set out to report the issue to the gem maintainers, following Richard’s suggestion:

…if you see a large memory use by a gem that you do need, please open up an issue with that library to let them know (be sure to include reproduction instructions). Hopefully as a community we can identify memory hotspots and reduce their impact. Before we can fix performance problems, we need to know where those problems exist.

I couldn’t find a dependency on iso3166 for phony_rails in my Gemfile.lock, but I did see the countries gem listed, and given that phony_rails deals with country codes and phone numbers, I deduced that the countries gem was the offender. A quick Google search for “ISO 3166”, as well as a search for “iso3166” in the countries GitHub repo provided confirmation.

While searching the countries GitHub issues, I noticed someone had already reported the large memory usage, so I added my findings to issue 230. At that point, I could have waited for the countries gem to be fixed (it eventually was about a month later), or attempted to address the memory issues myself. Instead, I took a step back to see how the countries gem was being used in phony_rails. Perhaps it would be easier to solve the memory problem by eliminating the dependency.

It turned out that the only reason the countries gem was needed was to map a country’s 2-letter ISO 3166 code to its calling code:

def self.country_number_for(country_code)
  ISO3166::Country[country_code.to_s.upcase].try(:data).
    try(:[], 'country_code')
end

Given that this information is static, and that new countries don’t get established every day, I submitted a pull request to the phony_rails repo to replace the countries dependency with a YAML file that contained the country code to number mapping. The pull request was accepted and merged, resulting in a 92% decrease in memory usage for the phony_rails gem (from 28.8MiB to 2.2MiB):

TOP: 92.332 MiB
  phony_rails: 2.2227 MiB
    phony: 2.1797 MiB

To produce the YAML file, I wrote a Ruby class that parsed the data in the countries gem, where each country has its own YAML file named after the country code, and inside each YAML file the numerical calling code is provided by the country_code key:

require 'yaml'

class CountryCodeToCallingCodeMapper
  def self.country_codes
    @codes ||= YAML.load_file('lib/data/countries.yaml')
  end

  def self.calling_codes
    country_codes.each_with_object({}) do |code, hash|
      hash[code] = {}
      calling_code = YAML.load_file("lib/data/countries/#{code}.yaml")[code]['country_code']
      hash[code]['country_code'] = calling_code
    end
  end

  def self.write_codes_to_yaml
    File.open('country_codes_to_calling_codes.yaml', 'w') do |f|
      f.write calling_codes.to_yaml
    end
  end
end

CountryCodeToCallingCodeMapper.write_codes_to_yaml

I encourage you to try out derailed_benchmarks in your project. You might discover memory issues similar to the one I found, that might lead to improving an open source project you use. Another takeaway from my experience is that in some cases, it might make more sense to implement your own solution rather than adding a heavy gem to your Rails app.