Spawning Multiple Threads at Heroku Using Parallel

Figure out the whys yourself. Here’s the how!

#lib/tasks/paralleltest.rb

# Q: Does spawning threads at heroku work?
task :paralleltest => :environment do
  res = "Who finished first? "
  Parallel.map 1..20, :in_threads => 4 do |nr|
    5.times { sleep rand; print "'#{nr}' " }
    puts "done with #{nr} "
    res += "#{nr} "
  end
  puts res
end

I’ve got the question in the rake task ” Q: Does spawning threads at heroku work? “. Well this is what running the rake task at heroku answered me

~/dev/app(master) $ heroku rake paralleltest --app myapp
(in /disk1/home/slugs/182581_156369f_19f1-5cf8dcaa-fc4b-4b21-a5e7-6fd85fad3e25/mnt)
'2' '4' '2' '3' '2' '1' '3' '4' '2' '1' '4' '3' '4' '4' done with 4 
'2' done with 2 
'1' '3' '5' '6' '3' done with 3 
'1' '7' '5' '6' '7' '1' done with 1 
'7' '6' '5' '6' '8' '7' '6' done with 6 
'5' '7' done with 7 
'10' '9' '8' '9' '5' done with 5 
'8' '10' '11' '8' '9' '8' done with 8 
'11' '10' '9' '12' '11' '10' '11' '9' done with 9 
'10' done with 10 
'11' done with 11 
'12' '14' '13' '13' '14' '13' '12' '15' '12' '15' '13' '15' '12' done with 12 
'14' '14' '16' '15' '14' done with 14 
'16' '13' done with 13 
'16' '17' '17' '18' '18' '16' '17' '16' done with 16 
'15' done with 15 
'19' '17' '19' '18' '17' done with 17 
'20' '20' '19' '19' '20' '20' '20' done with 20 
'18' '19' done with 19 
'18' done with 18 
Who finished first? 4 2 3 1 6 7 5 8 9 10 11 12 14 13 16 15 17 20 19 18 

I ran 4 threads on a 1 dyno app, using the parallel gem. The rand-function returns a number between 0 and 1, coincidences making ‘4’, ‘2’ and ‘3’ finish ahead of ‘1’. This is awesome and very useful. I love programming ruby!

About Ole Morten Amundsen

Developer, programmer, entrepreneur. Java, .Net, ruby, rails, agile, lean. Opinionated enthusiast!
This entry was posted in rails, ruby and tagged , , , , . Bookmark the permalink.

2 Responses to Spawning Multiple Threads at Heroku Using Parallel

  1. Max Williams says:

    I’m trying to use parallel inside one of my controller actions (in a rails 3 app), to push a long-running task into a new fork/thread, to prevent the action timing out the browser. I just want to use it something like

    #in controller
    @my_foo.save
    log “about to parallel”
    Parallel.map [1] do
    log “in parallel, about to do_long_task ”
    @my_foo.do_long_task
    log “done do_long_task”
    end
    log “after parallel”

    So that the controller action doesn’t wait for @my_foo to finish do_long_task before it renders out the response.

    However, when i do this, with extra logging, i can see that it’s not asynchronous – the log looks like this:
    “about to parallel”
    “in parallel, about to do_long_task ”
    “done do_long_task”
    “after parallel”

    Am i misunderstanding what parallel is actually for?

  2. Ole Morten Amundsen says:

    Yes, you misunderstand it a bit. What you send to map will be done in parallel threads. You send only one parameter, an integer. ‘Parallel.map [1]’ I sent 20. You need to put both the tasks that should run in parallell within the block.

    It would be simpler for you to just do Thread.new if you only have one task. You should look into it, but beaware, I do not recommend using threads if your not absolutely sure about the why and when. If one task is slow and the other fast, there is really not much gain; do it in sequence. Threading is difficult to test and to debug.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s