Lee Machin

Refinements under the knife

by

Surgeon Simulator 2013, courtesy of Edge Online

Refinements, made official in Ruby 2.1, are a way of ‘monkey patching’ objects without exposing those changes to other parts of the code. You would use them like this:

module RefinedString

  refine String do
    def to_thing
      # ...
    end
  end

end

class ThingWithRefinedString

  using RefinedString

end

Like most developers, I find the opportunity to abstract a language feature is often irresistible, and is a great way to learn about how something actually works. The question of whether or not the result is usable or useful is often irrelevant.

My hypothesis for such an abstraction was based on the idea of writing one-time refinements that you never intend to re-use, for which doing the module dance feels like introducing unnecessary boilerplate. This is what I thought I could do:

class ThingWithRefinedString

  patch String do
    def to_thing
      # ...
    end
  end

  def refined_string(str)
    str.to_thing
  end

end

Easy enough, right?

Wrong.

Refining refinements

Ruby’s an excellent language. It’s rarely the case that it doesn’t give you something to help you achieve what you want to do, no matter how awful or contrived that idea is. “You want to shoot yourself in the foot? Here’s all the rope you need,” says Ruby, rubbing its hands together in glee, knowing you can do a lot of things with a rope without shooting yourself. One of my previous quixotic experiments, node_module, is an excellent example of this.

When you call refine, you pass it a block containing all the methods you want to add or override for a certain class. It’s rare that you can’t substitute that for a proc, since the ability to pass blocks around as objects is what makes Ruby so wonderful for writing expressive and convenient DSLs, like we did ourselves with static_association and configurable_search.

Additionally, you have the ability to create anonymous modules. Combine these two together and you can theoretically create refinement modules at runtime.

module Patch

  def self.included(receiver)
    receiver.send :include, ClassMethods
  end

  module ClassMethods
    def patch(klass, &block)
      self.send :using, Module.new { refine klass, &block }
    end
  end

end

This mixin would presumably allow me to do exactly what I wanted, by calling using with an anonymous module containing my refinements. There are two reasons why this, or any attempt to do it any other sensible way, doesn’t work, and to understand that we can look at the Ruby source to see how refinements are implemented.

1. A block can’t be passed as a proc

“Wut? But it works everywhere else!” you might think. And there’s probably a reason why it doesn’t in this case (performance? subtle bugs?). A quick dive into eval.c shows us exactly why:

// ...

rb_block_t *block = rb_vm_control_frame_block_ptr(th->cfp);

// ...

if (block->proc) {
    rb_raise(rb_eArgError,
             "can't pass a Proc as a block to Module#refine");
}

Ruby is fetching the block that is currently being evaluated, and is actually making sure some clever clogs isn’t trying to pass in a proc. That at least means that doing this in pure Ruby isn’t gonna work, at least not that way.

My second attempt was in C, and after many hours poring through guides on how to write native extensions, and even more hours hopping from file to file in the Ruby source (thanks to GitHub’s search tool), I realised that none of the things I needed were exposed in the public API. Of course, after an excursion like that you never leave empty handed, and the insight was fascinating.

At the point of giving up, I remembered what I did with node_module: I could get the source of a block as a string (using sourcify) and then I could transform it into something that would work. Take THAT, interpreter.

require 'sourcify'

module Patch

  def self.included(receiver)
    receiver.send :include, ClassMethods
  end

  def self.refinements
    @refinements ||= {}
  end

  def self.new_refinement(receiver, class_to_refine, &block)
    refinement = Module.new
    refinement.module_eval <<-RB, __FILE__, __LINE__
      # Ruby doesn't complain when a refinement looks like this
      refine #{class_to_refine} do
        # get the source of the block, without the curly braces, and place it inline
        #{block.to_source(strip_enclosure: true, ignore_nested: true)}
      end
    RB

    module_name = "#{receiver}::#{class_to_refine}"
    self.refinements[module_name] = refinement
  end

  module ClassMethods
    def patch(klass, &block)
      class_exec Patch.new_refinement(self, klass, &block) do |mod|
        # mod here is our dynamically created refinement
        using mod
      end
    end
  end

end

This actually worked, and I was able to create refinements on the fly. But what about using them?

2. using cannot be called from within a method

They really don’t want you taking shortcuts with this feature. Even if you can find a way to create a refinement, the mere fact that the call to using occurs inside a method means it just won’t work. Check this out in eval.c:

if (prev_frame_func()) {
  rb_raise(rb_eRuntimeError,
           "Module#using is not permitted in methods");
}

Ruby is checking if the previous control frame is from inside a method, and if it is, you’re all out of luck. Since the key entry-point for this idea is a single patch method, you can’t really escape the fact you have to call it to do what you want.

Back to the drawing board

The original idea has been proven to be unworkable, but enough work has been done to try a different approach instead. Or a compromise, if you will. We can still hack a refinement into place, and maybe we can adapt it a little bit.

The blocker for the previous attempt was being unable to use using within a method. What happens if we build the abstraction around that instead?

Imagine that we’re now testing the ability to do this:

class ThingWithRefinedString

  using patched String do
    def to_thing
      # ...
    end
  end

  def refined_string(str)
    str.to_thing
  end

end

Seems doable, right?

Well… sort of.

Refactoring

Let’s first tweak the patch method and simplify it a little:

def patched(klass, &block) # was `patch`
  Patch.new_refinement(self, klass, &block)
end

We don’t care about refining and using in one fell swoop now, but we do have to worry about precedence. Will we actually receive a block, or will Ruby think we’re trying to pass it to using?

In the new example, the latter will be true. We don’t want that to happen, so we have to play with the syntax a bit and compromise on the prettiness to get what we want.

class ThingWithRefinedString

  using patched(String) {
    def to_thing
      # ...
    end
  }

  def refined_string(str)
    str.to_thing
  end

end

IT WORKS! Using the do ... end syntax for blocks doesn’t work, because Ruby thinks it’s being passed to using. Wrapping the first argument to patched in parentheses and using the alternative block syntax ensures the block is going exactly where we want it to.

And there we have it. Success! If you feel brave enough to use this in your own project, you can give the patched gem a shot.

Why so difficult?

I don’t know what the reasons are for going to such lengths to make this really hard to achieve. I’d love to be enlightened by any obliging reader, and to understand the reasoning behind restricting things so much, because for all the guns Ruby gives you to hang yourself, this is the first time I’ve not been able to shoot myself in the foot.