Beware Of EventMachine Periodic Timers
Sounds dramatic, right? I’m not saying you should stay away from them. There’s just one thing you need to know about them when you’re using EventMachine to spread out work across multiple threads.
EventMachine uses the concept of a reactor thread to handle distributing the work across a pool of worker threads. You’d normally use something like defer to tell EventMachine to take a thread from the pool and have it run that block.
When you declare a periodic timer, you also hand it over a block:
EM.add_periodic_timer(1) do
# some longer running task
end
But here’s the kicker: when that block is executed, EventMachine will call it in the reactor thread, even when you set up your periodic timer in a block that was called from defer, and even though you’d somehow expect EM to just do it.
Now, say you have multiple timers running, and one of them executes a longer running task. That task will block all the other timers from timing. Unless of course you’re using something like NeverBlock. If you rely on your timers being fired, that’s not great.
The solution is thankfully rather simple, you just have to defer again inside your periodic timer’s block. It’s not great, but it works. That way, the block will again be distributed to one of the threads from the pool.
EM.add_periodic_timer(1) do
EM.defer do
# some longer running task
end
end
For me that really doesn’t make any sense, and it’s not documented anywhere. Many props to Lourens helping me to find out the real problem. I know it’s not exactly related to AWS, but it suddenly becomes an issue when those long-running tasks are interactions with Amazon’s Web Services, especially when you have some calls that might take minutes which happens occasionally.
