Skip to content

Commit

Permalink
async http stack! going meta, query yourself.. to hit async mysql!
Browse files Browse the repository at this point in the history
  • Loading branch information
@igrigorik
igrigorik committed Jun 19, 2010
1 parent 72ea384 commit 6307f3f
Show file tree
Hide file tree
Showing 4 changed files with 26 additions and 1 deletion.
4 changes: 4 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.bundle
db/*.sqlite3
log/*.log
tmp/**/*
9 changes: 8 additions & 1 deletion Gemfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,19 @@ source 'http://rubygems.org'
gem 'rails', '3.0.0.beta4'

# async activerecord requires
gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony'
gem 'mysqlplus', :git => 'git://github.com/oldmoe/mysqlplus.git', :require => 'mysqlplus'
gem 'em-mysqlplus', :git => 'git://github.com/igrigorik/em-mysqlplus.git', :require => 'em-activerecord'

gem 'rack-fiber_pool', :require => 'rack/fiber_pool'

# async http requires
gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony/em-http'
gem 'em-http-request',:git => 'git://github.com/igrigorik/em-http-request.git', :require => 'em-http'
gem 'addressable', :require => 'addressable/uri'


# require 'em-synchrony/em-http'


# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'
Expand Down
12 changes: 12 additions & 0 deletions app/controllers/widgets_controller.rb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
class WidgetsController < ApplicationController
def index
Widget.find_by_sql("select sleep(1)")
render :text => "Oh hai"
end

def http
# going meta, query yourself, on the same thin server!
http = EM::HttpRequest.new("http://localhost:3000/widgets").get

This comment has been minimized.

# to view

Copy link
@jormon

jormon Apr 7, 2011

how would we catch an error here, if for instance we were trying to grab a url from an address/server that was not available? afaict, the only way would be something hacky like

failed = false
http = EM::HttpRequest.new(...).get
http.errback { failed = true }
if failed
    render : text => 'unreachable', :status => 500
else
    render :text => http.response
end

This comment has been minimized.

# to view

Copy link
@igrigorik

igrigorik Apr 7, 2011

Author Owner

You don't need the errback. Check the status of the http object directly once its returned.

ex: http.response_header.status

This comment has been minimized.

# to view

Copy link
@swistaczek

swistaczek Jun 9, 2012

Guys, is there any irc room that I can find you @igrigorik :)?
render :text => http.response
end
end
2 changes: 2 additions & 0 deletions app/models/widget.rb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
class Widget < ActiveRecord::Base
end

11 comments on commit 6307f3f

@maxwell
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey Ilya,

This looks great, it is awesome see all of these async parts working together.

Perhaps a silly question: with em-synchrony, do you recommend querying your own app thru a recursive call as to make sure the DB calls are properly wrapped in fibers, or did you just set it up this way so you could just test your server using your server :)

Thanks so much for the clarification.

@igrigorik
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to itself is more for the demo effect -- it shows that the same reactor can call on itself without causing a deadlock. In practice, calling on yourself would be kinda silly, since in theory, whatever action/controller you're calling.. you have access to in your Rails instance already. :-)

@shaokun
Copy link

@shaokun shaokun commented on 6307f3f Aug 5, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http = EM::HttpRequest.new("http://localhost:3000/widgets").get
render :text => http.response

I don't quite get this. I thought you need to do render within a callback block from EM::HttpRequest.
EM::HttpRequest won't block the thread, right? So, how can you get the http object and do render right away?

@igrigorik
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's the real nice thing of this setup. Every request is wrapped into a fiber, which means we can drop the callbacks and emulate a completely blocking API without blocking the thread or the Ruby runtime. The request is fired in async fashion, but em-synchrony resumes the execution once the callback fires. Take a look at em-synchrony: http://github.com/igrigorik/em-synchrony and also, this might be helpful: http://www.slideshare.net/igrigorik/no-callbacks-no-threads-cooperative-web-servers-in-ruby-19

@rolfb
Copy link

@rolfb rolfb commented on 6307f3f Sep 27, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would the difference be with using open-uri instead of EM::HttpRequest in this example?

@igrigorik
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you're running thin (app server), there is only one thread, so if open-uri blocks for 3 seconds while fetching data, then no other request will be served during that time.

@rolfb
Copy link

@rolfb rolfb commented on 6307f3f Sep 27, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the entire request isn't async?

@igrigorik
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean. open-uri is not, it uses net-http under the hood. If you use em-http + em-synchrony then you're fine.

@rolfb
Copy link

@rolfb rolfb commented on 6307f3f Sep 27, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies.
By request I meant the one hitting the Rack app.

Scenario would be 5 people hitting the app at the same time and an open-uri call in the controller layer.
Would each hit be blocking since Thin is 1 thread and open-uri isn't using the Fiber pool?

I'm guessing it would since that mostly resembles what you are saying, but I'd like to make sure. Thanks for answering this far.

@igrigorik
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Open-uri would block the main thread and that means the app-server would be unresponsive and could not process any other request. EM-http works around this by scheduling that fetch logic onto the Eventmachine reactor.

@rolfb
Copy link

@rolfb rolfb commented on 6307f3f Sep 27, 2010

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clearing that up. :)

Please # to comment.