My previous post was about using Thread in Rails which simply doesn’t work properly when you’re doing anything with ActiveRecord despite what anyone else is claiming.

This post will focus on my second attempt at a solution to intra-request threading in Rails. Basically, I have a Rails app where I want to run multiple computations at the same time. Since I had problems with Ruby’s native Thread method previously, I had no intention of going down that route again. I decided to try out the Spawn plugin.

Spawn was able to successfully segregate my multiple threads, or forks in my case. This got around the MySQL errors I reported in my previous post but it created a whole new set of problems all its own.

The first problem I had was that the spawn forks I was creating weren’t able to communicate their changes back to the main Rails request. I was creating forks and they were running fine and even saving data via ActiveRecord. I could confirm this via the console. The problem was that I need to know what data was being written by the spawn forks inside of the main Rails request. It was as if the main Rails request didn’t even know that something new was written to the database. I was able to get around this by forcing a reload from the database on the object I was trying to get association data from.

No big deal and it seems to make sense to do that anyway. That’s not where the real problem occurs though. After I got past that I realized that since each fork was essentially using a ‘copy’ of the database and not actually live connections that the data validations randomly failed. Basically if two forks are writing data to the database at the same time when there is a validates_uniqueness on the data. They both write to the database and both actually pass validation. The result is a database full of incorrect data which should have never passed the validation. I’m still not sure if that’s a problem with Spawn or an inherent problem with ActiveRecord’s connection pool.

I’ve been working on threading a Rails application lately and after reading headlines like ‘Rails is thread safe’ I figured how hard could it be. My first discovery was that when people talk about Rails and threads there are two different types of threading in Rails.

Multiple request threading – This is where Rails itself threads among different requests to your web server and allows ActiveRecord to behave properly without having to keep a copy of Rails in memory for each request.
Intra-request threading – This is where you have 1 request to your web server and inside the action you want to create multiple threads that run concurrently.

I’ll be talking about Intra-request threading. In particular, I want my threads to execute some code, read and write to the database, and play nicely with each other. My first attempt was to use the Ruby Thread method. This seemed to work somewhat until I started seeing strange errors coming in from MySQL. The problems seemed to occur randomly and what I determined was that the threads were trying to write to the database at the same time which ended up causing some collisions of sorts resulting in ‘lock wait timeout exceeded’ errors.

After considerable Googling, I found numerous posts about setting:

ActiveRecord::Base.allow_concurrency = true

The problem with that is that this is deprecated in the newest version of Rails in favor of connection pooling.

The short answer: Don’t use the Ruby Thread method within Rails when doing anything with ActiveRecord.