tag:blogger.com,1999:blog-32551071308633811392024-03-13T19:38:31.814+01:00RoR RamblingsJust some things from my work developing RoR applications.Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.comBlogger63125tag:blogger.com,1999:blog-3255107130863381139.post-13898809284175265652013-12-13T06:05:00.001+01:002013-12-13T06:05:55.976+01:00Ember.js and Rails - pt 3/x<h2>
Fixing ActiveModelSerializer in Ember-Data</h2>
Just a quickie... I need to write a longer post after the significant help I got yesterday from Ember dude <a href="https://twitter.com/cavneb" target="_blank">Eric Berry</a>.<br />
<br />
But I did manage to figure out using the DS.ActiveModelSerializer. There is an entry in the <a href="https://github.com/emberjs/data/blob/master/TRANSITION.md#rest-adapter-and-serializer-configuration" target="_blank">Transition Doc</a> of Ember Data talking about using underscores.<br />
It seems the DS.RESTAdapter now defaults to expecting camelCase'd class names. And the DS.ActiveModelSerializer does not fix this (thereby NOT working with Rails' ActiveModelSerializer out of the box - well done guys!). Also, the aforementioned document section is completely wrong.<br />
<br />
This is what you need to do somewhere (I put it in store.js) to get it to work with Rails' ActiveModelSerializer:<br />
<br />
<pre class="brush:js">App.ApplicationSerializer = DS.RESTSerializer.extend({
typeForRoot: function(root) {
var camelized = Ember.String.camelize(root);
var s = Ember.String.singularize(camelized);</pre>
<pre class="brush:js"> var deCs = Ember.String.decamelize(s);
// console.log(root + "; " + camelized + "; " + s + "; " + deCs);
return deCs;
}
});</pre>
<br />
I still need those nasty 'key:' entries on the rails side though to make associations work. But I am guessing it is something similar causing the problem.<br />
<br />
We're all in this together.Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-75359660627959504282013-12-10T07:51:00.001+01:002013-12-10T07:52:07.644+01:00Ember.js and Rails - pt 2/x<br />
Ok, it's been a couple of weeks (this is not the only thing I'm working on) and not too much progress. Getting my head around Ember has been a nightmare, not helped at all by the fact it is a moving target, especially in the case of Ember-Data.<br />
<div>
So much of a moving target that the whole thing shifted into RC mode.</div>
<div>
So, a lot of changes have had to be made.</div>
<div>
<br /></div>
<h2>
Die, Coffeescript, Die Die Die.</h2>
<div>
<br /></div>
<div>
The other thing is I cannot express deeply enough my hatred of Coffeescript. It seems, well, pointless. It seems to be used to write Javascript and I really don't see any advantages over just writing Javascript. It is argued that it is smaller, but that is not really the point when it is unreadable.</div>
<div>
It is like I say that I write my letters in Japanese, which will eventually be converted to English, because in Japanese I can use a single Kanji character to express a whole word or concept in English. The fact that you have to learn Japanese to accomplish something you could adequately do from the start is the killer.</div>
<div>
<br /></div>
<div>
So, I have ditched Coffeescript for now.</div>
<div>
<br /></div>
<h2>
Ember Store</h2>
<div>
So, the store.js file now looks like this:</div>
<div>
<br /></div>
<div>
<div>
<pre class="brush: js">HorseFeeder.ApplicationAdapter = DS.RESTAdapter.extend({
namespace: 'api/v1'
});
</pre>
<div>
<br />
<b>Lesson #1 - don't use DS.ActiveModelSerializer</b><br />
<br />
If I put this line in:<br />
<br /></div>
<div>
<pre class="brush: js">HorseFeeder.ApplicationSerializer = DS.ActiveModelSerializer.extend({});</pre>
<br />
everything stops parsing. Even though I am using ActiveModelSerializer on the Rails side.<br />
<br />
<h2>
Routes</h2>
Routing has changed in Ember-Data. It's a little simpler, but, they got rid of the store object, which means that you can't call the store directly from Firebug.<br />
<br />
Routes now look like this:<br />
<br />
<br />
<pre class="brush: js">HorseFeeder.EventDaysIndexRoute = Ember.Route.extend({
model: function() {
return this.store.find('event_day');
}
});
HorseFeeder.EventDaysShowRoute = Ember.Route.extend({
model: function(params) {
obj = this.store.find('event_day', params.event_day_id);
return obj;
}
});
</pre>
<br />
<h2>
Serializers</h2>
<div>
Right now it looks like this (see 'current crisis points' below):</div>
<div>
<br /></div>
<pre class="brush: ruby">class EventDayIndexSerializer < ActiveModel::Serializer
attributes :id, :day
has_many :events, embed: :ids, key: :events
end </pre>
<br />
Note that if I don't do that key: :events thing Ember doesn't see the objects.<br />
<h2>
Current Crisis Points</h2>
</div>
</div>
<div>
Right now the backend is spewing out data and the front end is taking it. i.e. I am doing what all the Ember tutorials on the web do. But that is not the real world. Here are a list of my current pain points</div>
<div>
<br /></div>
<div>
<ol>
<li>When I send out a list of objects in an index call, I do not want to send out all their associations/children because that would mean basically dumping the whole database, right?! As all objects are related to each other in my model it is unmanageable and slow. I am fixing this right now by writing serializers that only dump out the object, not the children.</li>
<li>If you do what I do above (with the limited data at the index level) I cannot figure out how to get Ember to reload the object on the 'show' so I can fill in all the data. To me this seems to be such an obvious way to do it I can't believe I've spent 2 days trying to get it to work.</li>
<li>What doesn't the ActiveModelSerializer in Ember work for AMS in Rails?</li>
<li>The pain and time so far have caused me to question the wisdom of using Ember!</li>
</ol>
<div>
I have a few open StackOverflow questions... so hopefully my next instalment will be all sweetness and light.</div>
</div>
<div>
<br /></div>
</div>
Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-56093690311113448482013-12-10T07:15:00.001+01:002013-12-10T07:52:07.642+01:00Learning Ember.js and Rails Part 1 (of many!)<!--?xml version="1.0" encoding="UTF-8" standalone="no"?-->
<span style="font-family: Arial;">I’m starting on a new, massive, application. I don’t like to make things easy on myself so instead of doing this in my usual RoR/jQuery stack I am jumping in and doing this with:</span><br />
<div style="font-family: Arial;">
<ul>
<li>Ruby 2</li>
<li>Rails 4</li>
<li>Ember.js</li>
</ul>
<div>
This presents some challenges, not the least of which is Ember.js. I’ve never used a front end framework before. But I can see the power of it, plus, one of the key features of this new app is that it will provide a data api. With Ember.js, I believe, almost all interaction with the data will be through an api, so it seems like a good start and a way to sanity check what is coming in and out to other consumers of this api.
</div>
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
I know I suck at writing blog posts, but, I will try and document this journey because I have looked around and there doesn’t appear to be too many resources for people doing this. Everything I have found is much too simple and is not for the experienced Rails developer.
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
I did find one good resource from Tony Coconate (<a href="http://www.devmynd.com/blog/2013-3-rails-ember-js">http://www.devmynd.com/blog/2013-3-rails-ember-js</a>) and I am starting off from that. Now, that may or may not be a good idea!
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
<b>Setup</b></div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
I set up a new rails app. Then chucked some stuff into the Gemfile:
</div>
<pre class="brush:ruby"># front end
gem 'ember-rails'
gem 'ember-source'
gem 'ember-auth-rails'
gem 'ember-auth-source'
</pre>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
<b>Create Resources</b></div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
After, my first steps were to all but ignore the front end.
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
I created my models as resources, like in Tony’s article. For example:
</div>
<div style="font-family: Arial;">
<br /></div>
<pre class="brush:bash">rails g resource Trainer name:string additional_info:text provider:references provider_key:string —no_helper</pre>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
This means you end up with:
</div>
<div style="font-family: Arial;">
<ul>
<li>a controller, </li>
<li>a model</li>
<li>a serializer - from active model serializer</li>
<li>a db migrate script </li>
<li>two coffee script views (trainer and trainers)</li>
<li>entries in routes.rb</li>
</ul>
<div>
<b>Moving Controllers</b></div>
<div>
<br /></div>
<div>
Again, as per Tony’s article, I am going to use Ember Data (whether or not it is ready for prime time now is not too important. This app won’t see the light of day for months yet). So, I move all the controllers from /app/controllers to /app/controllers/api/v1 and put Api::V1:: at the front. I also put in standard actions. So now my controller looks like this:
</div>
</div>
<div style="font-family: Arial;">
<br /></div>
<pre class="brush:ruby">class Api::V1::TrainersController < ApplicationController
respond_to :json
def index
respond_with Trainer.all
end
def show
respond_with Trainer.find(params[:id])
end
end
</pre>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
<b>Modify Routes</b></div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
I also modify routes.rb to look like this:
</div>
<div style="font-family: Arial;">
<br /></div>
<pre class="brush:ruby">namespace :api do
namespace :v1 do
resources :trainers
end
end</pre>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
At this point I can point my browser at localhost:3000/api/v1/trainers.json and get a list (assuming I have data!) or to localhost:3000/api/v1/trainers/1.json to get the single record.
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
So far so good.
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
In my mind the data side of the app is on the way. I will not worry about any more Rails stuff for now.
</div>
<div style="font-family: Arial;">
<br /></div>
<div style="font-family: Arial;">
Next is learning Ember.js </div>Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-44126108976972575432013-09-25T17:22:00.000+02:002013-09-25T17:22:00.337+02:00HTTP Basic authentication with Rails for iOSRecently I've been getting into iOS and developing a interesting iOS app (more info coming soon!)<br />
<br />
But, as the app is all about showing data of course I needed to create an API in my rails app to feed that data. But, I didn't want someone else to be able to grab it.<br />
<h2>
The Rails App Side</h2>
<h3>
Configure a custom MIME type</h3>
<div>
First thing I did was set up a custom mime type in initializers/mime_types.rb:</div>
<div>
<div>
<pre class="brush:ruby">Mime::Type.register_alias "text/json", :theapp</pre>
</div>
</div>
<div>
That let me call urls like /give_me_data.theapp</div>
<h3>
Add the 'respond_to'</h3>
<div>
In the applicable controllers add a respond_to (well, I already had one, so add to it):</div>
<div>
<div>
<pre class="brush:ruby">respond_to :html, :xml, :theapp</pre>
</div>
</div>
<h3>
Make templates with json_builder</h3>
<div>
I luuuvvvvv the <a href="https://github.com/dewski/json_builder" target="_blank">json_builder</a> gem. So I created a whack of templates with the .json_builder extension. Because of the mime type setting it would automatically get called.</div>
<h3>
Add the http_authentication stuff</h3>
<div>
I am a little bit lazy, so, I just chuck this into my application_controller. That means that all :theapp requests need authentication:</div>
<div>
<br /></div>
<div>
<div>
<pre class="brush:ruby">protected
def http_basic_authentication
if request.format == :ios
authenticate_or_request_with_http_basic do |username, password|
username == 'myAmazingApp' && password == 'theBestestPasswordEver'
end
end
end</pre>
</div>
</div>
<h2>
The iOS app side</h2>
<h3>
The URLs</h3>
<div>
Simply chuck your authentication stuff into your urls:</div>
<div>
<pre class="brush:cpp">NSString *const RegionDataBaseURL = @"http://myAmazingApp:theBestestPasswordEver@localhost:3000/regions/%@.theapp";
</pre>
</div>
<h3>
<span style="color: black;">Pulling...Data</span></h3>
<div>
<span style="color: black;">I do my stuff like this (thanks to Ray Wenderlich):</span></div>
<div>
<pre class="brush:cpp"> dispatch_async(kBgQueue, ^{
NSURL *url = [NSURL URLWithString: [NSString stringWithFormat:RegionDataBaseURL, [self.region permalink]]];
NSData* data = [NSData dataWithContentsOfURL: url];
[self performSelectorOnMainThread:@selector(fetchedRegionData:)
withObject:data waitUntilDone:YES];
});</pre>
</div>
<div>
Yes, I could also put the username and password in some other place and sub it into the URL, but brute force works for now.</div>
<div>
<span style="color: black;"><br /></span></div>
<div>
<span style="color: black;">Like I say, I am starting out in </span>iOS so maybe this is too noddy for all you pros, but, it works.</div>
Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-70830887154475451982013-09-24T15:16:00.000+02:002013-09-24T15:16:28.972+02:00The gotcha's in Node.js - or how I reduced our bandwidth by 75%So, I have this great application that distributes, in pretty close to real time, information across hundreds of screens in Germany. It does this with a combination of Rails, Node.js and HTML/Javascript.<br />
But we noticed MASSIVE traffic.<br />
My model was to pump out the data and have intelligent pages decide what to show.<br />
It turns out this is very inefficient.<br />
<br />
<h3>
Not ready for primetime</h3>
We have 4 'channels' of information. The software had originally been built on a 'tv station' idea. And, in TV, everything gets pumped down the line (or over the air) and then the tv 'tunes in' to a channel and shows the appropriate content.<br />
In our case, we had a 'screen structure' message that was sent out once a minute and then in between each element on the screen would send out its own updates if it was altered.<br />
It was the screen structure message that was the problem.<br />
<h3>
The old way</h3>
I was doing something like this on the page:<br />
<br />
<pre class="brush:js;">client.subscribe('/source', function(message) {
window.setTimeout(function() {
process_source_message(message);
}, 1);
})
</pre>
<br />
And in Node app.js I was doing this:<br />
<pre class="brush:js;">app.post('/source_state', function(req, res) {
bayeux.getClient().publish('/source', req.body);
res.send(200);
});
</pre>
<br />
And in Rails, something like this:<br />
<pre class="brush:ruby;">RestClient.post [NODE_CONFIG[Rails.env]['node_url'], NODE_CONFIG[Rails.env]['source_update_path']].join('/'), JSON(json)</pre>
<br />
So, every page would get every 'source' message.<br />
<h3>
The new way</h3>
I realised that I should have the pages listen to their specific channel. I had got hung up in the past on the idea that I didn't know the name of the source in advance - it could be anything as it is user defined.<br />
<br />
But, inside of the source JSON was already this:<br />
<pre class="brush:ruby;">json = {
"source" => {"name" => source.slug}
}
</pre>
<div>
<br /></div>
It turns out I didn't need to change the back end (Rails) at all.<br />
<br />
In the app.js I just needed<br />
<pre class="brush:js;">app.post('/source_state', function(req, res) {
bayeux.getClient().publish('/'+req.body.source.name, req.body);
res.send(200);
});
</pre>
<br />
<br />
<br />Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-47306727990789334702011-10-30T14:13:00.002+01:002011-10-30T14:15:01.538+01:00installing sunport_rails on ubuntu productionWe've added search functionality to Toygaroo with Solr via sunspot_rails.<div><br /></div><div>If you are going to install it on Ubuntu then make sure you do:</div><div><br /></div><div><div><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>sudo apt-get install openjdk-6-jdk</code></pre></div><div></div></div><div><br /></div><div>First!</div>Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-37654641103603775432011-10-01T15:15:00.003+02:002011-10-01T15:18:59.418+02:00Using the html5 boilerplate and rails 3I am experimenting with using the <a href="http://html5boilerplate.com/">html5 boilerplate</a> with rails 3.0x. It is an interesting setup. I am using Russ Frisch's html5 boilerplate template, but updating it to use the latest boilerplate code.<div>It seems to be a pretty nice way of getting your css and js ducks in a row. Over the next few days I need to see how it handles having jquery mobile in the mix. I am also not 100% sure how to use a some sort of grid layout template (like 960 or 114opx) with it. Or even if you should! </div><div>We shall see.</div>Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-32036413803185654252011-09-28T11:44:00.003+02:002011-09-28T11:48:47.008+02:00Suffering Rails3 slowdownWe've just pushed our Rails 3 upgraded app to production... and are suffering a massive slowdown in insert/update speed over Rails 2.<div>At the moment I am not sure of the<span class="Apple-tab-span" style="white-space:pre"> </span>exact cause of this. </div><div><br /></div><div>It *might* be mysql inserts, though I can't quite see why that would be. </div><div><br /></div><div>It *might* be because this new version we are using vestal_versions to track changes.</div><div><br /></div><div>It might be because the moon is in the house of Mars for all I know!</div><div><br /></div><div>I hate getting stung by unknowns. The speed on our test environment is tolerable, slightly slower than the rails 2 version, but I was willing to accept that because the new version is doing so much more.</div><div><br /></div><div>Bench-marking is one thing.. but know why the bench marks are slower is the key!</div>Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-37309259306645781552011-09-26T19:44:00.002+02:002011-09-26T19:47:42.532+02:00Expiring fragments from daemonsWe have an application that gets its data from a series of daemons that go out and read in data. This works great, except, we are caching pages. And I'd like to expire those pages based on an update.<br />It turns out that an Observer doesn't have access to expire_action or fragment. And a Sweeper is not called from data-only (i.e non-controller) updates! Buggers!<br /><br />But there is a solution. You can call the sweeper directly from your importer:<br /><br />MySweeper.instance.clean_up(model_instance)<br /><br />This works, except I couldn't get it reliably to expire the actions. So, I used direct calls to Rails.cache.delete to do this.<br /><br />Thinking about it, I guess I could then have just written an observer! As those do get called from controller-less updates.Philhttp://www.blogger.com/profile/06847937813473558958noreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-7850200577870275862011-07-21T10:45:00.005+02:002011-07-21T10:54:59.454+02:00meta_search sort_link helper and associationsIt took me a while to find this, so, for my own memory I am going to quickly write this up.<br /><br />I have a view that shows a table of objects (a pretty standard index view). The only issue was I want to sort on one of the columns that actually has data coming not from the main object but an association.<br />What I discovered is a line in a posting <a href="http://metautonomo.us/2010/11/21/metasearch-metawhere-and-rails-3-0-3/">here</a> that says:<br /><br /><span style="font-style:italic;">You can define your own custom sort scopes. Define scopes named “sort_by__asc” and “sort_by__desc” and sort_link @search, :name will work as you might expect.</span><br /><br />So, I have an object of 'info' defined like this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class Info < ActiveRecord::Base<br /> belongs_to :region<br /> scope :contains_string, lambda {|str| where(:name.matches % "%#{str}%")}<br /> search_methods :contains_string<br /><br /> scope :active, lambda{where(:active => true)}<br /> search_methods :active, :type => :boolean<br />end<br /></code></pre><br /><br />What I found is that you can put this at the end:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>scope :sort_by_region_name, lambda{joins(:region).order("regions.name asc")}<br />search_methods :sort_by_region_name<br /></code></pre><br /><br />and then in my view I can do:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><table><br /> <thead><br /> <tr><br /> <td width="25%"><%= sort_link @info_search, :name, "Info" %></td><br /> <td width="25%"><%= sort_link @info_search, :region_name, "Region" %></td><br /> <td><%= sort_link @info_search, :active %></td><br /> </tr><br /> </thead><br /> ...body info...<br /></table<br /></code></pre><br /><br />And in my controller:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class Admin::InfosController < Admin::BaseController<br /> def index<br /> search = params[:search] || {"meta_sort" => "name.asc"}<br /><br /> @info_search = Info.search(search)<br /> @infos = @linfo_search.paginate(:page => params[:page]||1, :per_page => 15) # load all matching records<br /> end<br />end<br /></code></pre><br /><br />And presto I have an index that<br />a) default to sorting by name (see the first line of the index method)<br />b) let's me sort on an associated value<br /><br />Cool!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-38839147971956233662011-07-01T14:50:00.003+02:002011-07-01T14:53:58.431+02:00I'm sorry, but POW sucksI have been (trying to) use Pow, and the Powder gem, for a few weeks now. At the outset it looked good: you get a local domain for you test on, you don't have to worry about deployments, etc etc.<br />But it sucks.<br />If you get an error in rendering (and we are using this in development, so of course this is going to happen) pow goes into a tail spin and on my MacPro it takes almost a minute to come out of it as it retries the request dozens of times.<br /><br />So you end up with a huge log that looks pretty much like this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code> SQL (2.2ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (0.9ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (0.9ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (1.0ms) SHOW TABLES<br /> SQL (1.3ms) SHOW TABLES<br /> SQL (0.9ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (0.9ms) SHOW TABLES<br /> SQL (1.4ms) SHOW TABLES<br /> SQL (1.2ms) SHOW TABLES<br /> SQL (1.4ms) SHOW TABLES<br /> SQL (1.1ms) SHOW TABLES<br />Rendered /Users/smyp/.rvm/gems/ree-1.8.7-2011.03@toygaroo_r3/gems/actionpack-3.0.3/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (8663.2ms)<br />Rendered /Users/smyp/.rvm/gems/ree-1.8.7-2011.03@toygaroo_r3/gems/actionpack-3.0.3/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (8728.0ms)<br />Rendered /Users/smyp/.rvm/gems/ree-1.8.7-2011.03@toygaroo_r3/gems/actionpack-3.0.3/lib/action_dispatch/middleware/templates/rescues/template_error.erb within rescues/layout (22759.6ms)<br />Rendered /Users/smyp/.rvm/gems/ree-1.8.7-2011.03@toygaroo_r3/gems/actionpack-3.0.3/lib/action_dispatch/middleware/templates/rescues/template_error.erb within rescues/layout (22758.7ms<br /></code></pre><br /><br />I am going back to using Passenger standalone. Ok, you loose the pretty local domain, but it works much more reliably.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-67590321178982506932011-05-02T15:10:00.002+02:002011-05-02T15:12:38.401+02:00nginx, rails and ubuntu - 502 bad gatewayWe were getting tons of 502 errors under load, but then I stumbled across a posting in a news group.<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>cat /proc/sys/net/core/somaxconn<br /></code></pre><br /><br />Will show you how many connections you can have. This should be 1024, because Phusion Passenger is hard coded for this value. Mine was 128!<br /><br />Do this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>sudo sysctl -w net.core.somaxconn=1024<br /></code></pre><br /><br />And then restart nginx.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3255107130863381139.post-46161616900840417002011-05-01T10:24:00.003+02:002011-05-01T10:32:00.687+02:00get-flash-videos and osx (off topic)I travel a lot and want to watch tv shows from the UK while I am doing so. There is get_iplayer that works nicely to get BBC shows, but on occasion there are things on itv (!) I watch. For this I use 'get-flash-videos'.<br /><br />I had a tough time getting this working on OSX, so in a nutshell here is what I did:<br /><br />1) Get 'get-flash-videos'<br />The <a href="http://code.google.com/p/get-flash-videos/">home page</a> is here. You pull that down somewhere onto your mac (I have it in a Downloads/get-flash-videos directory).<br /><br />2) Update perl!<br />This is the key! I know nothing about perl, but here is what I did:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>perl -MCPAN -e shell<br /></code></pre><br />This brings up a perl shell. Perl seems to have a package manager called CPAN. You will need to update this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>install Bundle::CPAN<br /></code></pre><br /><br />Then you need to install Digest:SHA and Compress:ZLIB:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>install Digest::SHA1<br />install Compress::Zlib<br /></code></pre><br /><br />3) FLVStreamer and rtmpdump<br />Installing FLVStreamer is non-trivial and there are other guides about that. <br />I download rtmpdump from <a href="http://www.smallsoftware.co.uk/downloads/">here</a><br />Then<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>chmod +x rtmpdump<br />sudo cp rtmpdump /usr/local/bin/.<br /></code></pre><br /><br />4) Grab video!<br />now you should be able to do something like:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>./get_flash_videos http://www.itv.com/itvplayer/video/?Filter=228293<br /></code></pre><br />And have it pull down an mp4. I use Handbrake to convert it for playing on the ipad.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-17965013900435745962011-03-28T15:58:00.003+02:002011-03-28T15:58:00.146+02:00Toygaroo on RailsSeeing as there has been a lot of press recently about my company <a href="http://www.toygaroo.com">Toygaroo</a> I thought I might throw out some tech info for those of you who care.<br />For those who don't know, Toygaroo is America's biggest toy rental company (think netflix for kid's toys). Recently - March 25th, 2011 - we appeared on the season premiere of Shark Tank - a national television show on ABC. About 4.6 million people watched the show.<br /><br /><span style="font-weight:bold;">The Platform</span><br />Toygaroo is a Ruby on Rails 3 application. It is based heavily on the code we wrote for FilmAmora.com, Spain's leading DVD rental company. We are running on Ubuntu 10.04 LTS. We're using <a href="http://www.modrails.com/">Passenger 3</a> and <a href="http://wiki.nginx.org/">Nginx</a> (doesn't everyone?!). We're using <a href="https://github.com/mperham/dalli">Dalli</a> in front of MemCached (though we've had some issues with this that I really should blog about!).<br /><br /><span style="font-weight:bold;">The Host</span><br />Right now we are running on Amazon EC2 service - though with <a href="http://www.pcmag.com/article2/0,2817,2382636,00.asp">Mark Cuban</a> coming on board that might change. I am a big fan of EC2, though I think the machines are a little underpowered for what you pay.<br /><br /><span style="font-weight:bold;">The numbers!</span><br />In the 2 hours after the show aired we received around 70,000 page views. The basic architecture is a load balancer sitting in front of a whack of app servers. We are not using a scaling solution right now - hey, we're a start up! - so we wind up more servers if we feel we need them. It is a pretty simple process - I have a script for us to follow to get an Ubuntu server up and running in no time.<br /><br /><span style="font-weight:bold;">Caching</span><br />I looked into other solutions, like Varnish, but decided that Rails could handle the job with a combination of page, action and fragment caching. And I haven't been wrong so far. Even under heavy load we are getting great response time. The key - as I found out with FilmAmora - is what level to cache on. We cache 'blocks'. i.e. if you look at an index page with lots of toys we cache each toy block. That block can appear on many different pages, so it is a nice solution I find.<br /><br />As time goes on I'd like to post more about how Toygaroo is coded and running as I think it will provide a nice real world example for what you can do with Rails 3. If you have any questions drop me a line (comment on here).Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-61072753630529837712011-02-12T11:08:00.011+01:002011-02-12T11:39:33.548+01:00Background File Processing Daemon in RubyI am writing this up because I scoured the net and could not find what I would have thought would have been a common thing to do.<br />We have an application that needs to watch several directories (on the server) and parse files that are placed there (via scp) by a third party. FWIW, these files represent sports betting prices. <br /><br /><span style="font-weight:bold;">Requirements</span><br /><br />A background task that could be <br />* monitored<br />* run forever!<br />* process files instantly - parse them into ruby objects and store them into our database for use by the rails app<br /><br />Our Rails app is written in 2.3.x (its been running for a while) and uses Bundler.<br /><br /><span style="font-weight:bold;">The Solution</span><br /><br />After some poking around I decided to use a combination of the <a href="http://daemons.rubyforge.org/">Daemons</a> gem, <a href="http://www.railsfire.com/article/ruby-eventmachine-short-introduction">EventMachine</a> and <a href="https://github.com/ttilley/fssm">FSSM</a>.<br /><br />The Daemon<br /><br />This was inspired heavily by a posting on StackOverflow.<br /><br />1) Install what you need<br />I tried to get this working successfully with Bundler, but it was a no go. So I needed to install daemons and eventmachine 'normally':<br />sudo gem install daemons eventmachine fssm<br /><br />2) Setup the Daemon:<br /><br /><span style="font-style:italic;">Setup</span><br /><br />Usual stuff for a ruby file:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>#!/usr/bin/env ruby<br />require 'rubygems'<br />require 'daemons'<br /></code></pre><br /><br />We have multiple directories that need watching. So have an array:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><br />watch = [<br /> "/Users/smyp/development/wl/xtf/horse", <br /> "/Users/smyp/development/wl/xtf/sport",<br /> "/Users/smyp/development/wl/xtf/live",<br /> "/Users/smyp/development/wl/xtf/alpha"<br />]<br /><br />if ENV['RAILS_ENV'] == 'production'<br /> watch = ["/home/mcdata/horse", "/home/mcdata/sport", "/home/mcdata/live", "/home/mcdata/alpha"]<br />end<br /></code></pre><br /><br />We launch a separate daemon for each directory as we don't want a huge file in the horses directory to slow down processing in the live directory.<br /><br /><span style="font-style:italic;">Daemon Config</span><br /><br />With the daemons gem you can set things like what the process will be called. And where the pid file will reside, etc etc.<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>dir = File.expand_path(File.join(File.dirname(__FILE__), '..'))<br /><br />daemon_options = {<br /> :app_name => "xturf_file_monitor",<br /> :multiple => false,<br /> :dir_mode => :normal,<br /> :dir => File.join(dir, 'tmp', 'pids'),<br /> :backtrace => true<br />}<br /></code></pre><br /><br /><span style="font-weight:bold;">3) The Actual Daemon</span><br /><br />Cue spooky music!<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class PriceDaemon<br /> attr_accessor :base_dir<br /> def initialize(base_dir)<br /> self.base_dir = base_dir<br /> end<br /><br /> def dostuff<br /> logger.info "About to start job for #{base_dir}"<br /> EventMachine::run {<br /> # Your code here<br /> xhj = PriceFileJob.new(base_dir)<br /> xhj.clear_backlog<br /> FSSM.monitor(base_dir) do<br /> create {|base, relative| xhj.clear_backlog}<br /> update {|base, relative| xhj.clear_backlog}<br /> end<br /> }<br /> end<br /><br /> def logger<br /> @@logger ||= ActiveSupport::BufferedLogger.new("#{RAILS_ROOT}/log/price_file_monitor.log")<br /> end<br />end<br /></code></pre><br /><br />What this does is:<br />a) create a class that takes the directory to watch as an initialize parameter<br />b) do an EventMachine run that first clears out any backlog files then fire up an FSSM monitor. The FSSM monitor gives us events on create, update (and delete, but we don't care about that). As a safety measure I simply trawl through the entire directory every time a file is created or updated. This ensures that anything we missed will get caught. <br />We delete files ourselves after processing, so the directory should only have a few files in it anyway.<br /><br /><span style="font-weight:bold;">4) Spawn the Daemon</span><br /><br />Bring on Mia Farrow!<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>watch.each_with_index do |base_dir, i|<br /> Daemons.run_proc("price_daemon_#{i}", daemon_options) do<br /> Dir.chdir dir<br /> PriceDaemon.new(base_dir).dostuff<br /> end<br />end<br /></code></pre><br /><br />This will go through our array and file up a daemon for each directory. There are downsides to doing it this way - its not so easy to start and stop one (but then they shouldn't ever die, so if they do we just start and stop them all).<br /><br /><span style="font-weight:bold;">5) The File Processor</span><br /><br />This of course will be specific to your operation, but, here's an outline of ours:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class PriceFileJob<br /> attr_accessor :base_dir<br /> def initialize(base_dir)<br /> self.base_dir = base_dir<br /> logger.info "watching #{base_dir}"<br /> end<br /><br /> def logger<br /> @@logger ||= Logger.new("#{RAILS_ROOT}/log/price_file_job_#{base_dir.split("/").last}.log", "daily")<br /> end<br /><br /> def clear_backlog<br /> files = Dir.new(base_dir).entries.sort_by{|c| File.stat(File.join(base_dir, c)).ctime}<br /> files.each do |file|<br /> process_file(file)<br /> end<br /> end<br /> <br /> def process_file(file)<br /> end<br /> <br /> private<br />end<br /></code></pre><br /><br /><span style="font-weight:bold;">6. Capistrano</span><br /><br />We use Capistrano to deploy, so I included some tasks in our deploy.rb<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>before "mc:release", "file_processors:stop"<br />after "mc:release", "file_processors:start"<br /><br />namespace :file_processors do<br /> desc "start processors"<br /> task :start, :roles => :db do<br /> run "cd #{current_path}; RAILS_ENV=#{fetch :rails_env} ruby ./script/price_file_monitor.rb start"<br /> end<br /><br /> desc "get status of processors"<br /> task :status, :roles => :db do<br /> run "cd #{current_path}; RAILS_ENV=#{fetch :rails_env} ruby ./script/price_file_monitor.rb status"<br /> end<br /><br /> desc "stop processors"<br /> task :stop, :roles => :db do<br /> run "cd #{current_path}; RAILS_ENV=#{fetch :rails_env} ruby ./script/price_file_monitor.rb stop"<br /> end<br />end<br /></code></pre><br /><br />That's it! I hope you found this interesting.<br /><br />I should also write up how we monitor these processes... maybe next time!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-14070104152643727182010-11-24T10:34:00.004+01:002010-11-24T10:45:41.293+01:00will_paginate and ajax in rails 3After googling and stackoverflowing around I couldn't find anything that explicitly said how do to this.<br /><br />I have a page that has 'pagable' areas on it, and want to load these using ajax. And I am in rails 3!<br /><br />I found it actually was quite easy and only needed a few lines of code.<br /><br /><span style="font-weight:bold;">1 - In the page view (users/show.html.erb)</span><br />To enable the data-remote attribute on will_paginate's links<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><div id="queue"><%= render :partial => 'queue' %></div><br /><script><br />$(document).ready(function() {<br /> $('.pagination a').attr('data-remote', 'true');<br />});</script><br /></code></pre><br /><br /><br /><span style="font-weight:bold;">2 - In the controller (users_controller.rb)</span><br />To allow a js response<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><br />def show<br /> @user = User.find(params[:id])<br /> @queue_items = @user.queue_items.with_state(:pending).paginate(:page => params[:queue_page] || 1, :per_page => 1)<br /> respond_to do |format|<br /> format.html <br /> format.js<br /> end<br />end<br /></code></pre><br /><br /><br /><span style="font-weight:bold;">3 - A show.js.erb template</span><br />You need the second line to reapply the data-remote to the new links<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>$('#queue').html('<%=escape_javascript render :partial => "queue" %>');<br />$('.pagination a').attr('data-remote', 'true');<br /></code></pre><br /><br /><br /><br />That's it! Your new dataset should load into the 'queue' div.Unknownnoreply@blogger.com7tag:blogger.com,1999:blog-3255107130863381139.post-41611868120863703912010-10-18T10:46:00.005+02:002010-10-18T11:03:45.014+02:00Customizing Devise to a pseudo multi-stage Signup<span style="font-weight:bold;">The Requirements!</span><br />On our new site we want to have a kind of multi-stage signup. The first page the user chooses a type of subscription, the second page they create their account, with address information, and the third page they enter their billing info. If they don't do the third stage that's ok as we will warn them that their account is incomplete (and they won't get any services until they give us the billing info)<br /><br />We are doing this new site in Rails3 and I thought I'd use <a href="http://github.com/plataformatec/devise">Devise</a> as the authentication engine. In our old site we use Restful Authentication, which has been awesome. But <a href="http://github.com/plataformatec/devise">Devise</a> seems to be the thing all the kids are into today!<br /><br />So here's how we did it:<br /><br /><span style="font-weight:bold;">Step 1:<br />Create a New Controller<br /></span>We created a new controller to handle the first stage of the signup. It's very simple. It only is a "new" method (for now) as we do no saving or updating of the subscription.<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>rails g controller subscriptions new<br /><br /> create app/controllers/subscriptions_controller.rb<br /> route get "subscriptions/new"<br /> invoke erb<br /> create app/views/subscriptions<br /> create app/views/subscriptions/new.html.erb<br /> invoke rspec<br /> create spec/controllers/subscriptions_controller_spec.rb<br /> create spec/views/subscriptions<br /> create spec/views/subscriptions/new.html.erb_spec.rb<br /> invoke helper<br /> create app/helpers/subscriptions_helper.rb<br /> invoke rspec<br /> create spec/helpers/subscriptions_helper_spec.rb<br /><br /></code></pre><br /><br />The controller looks like this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class SubscriptionsController < Devise::RegistrationsController<br /> def new<br /> @subscription_plans = SubscriptionPlan.visible<br /> end<br />end<br /></code></pre><br /><br />Pretty basic stuff!<br /><br />The view is:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><% title "Sign up" %><br /><br /><div>Pick a subscription</div><br /><% semantic_form_for :subscription, :url => users_sign_up2_path, :html => { :method => :get } do |form| %><br /> <%= form.input :plan_id, :as => :radio, :collection => @subscription_plans.map{|sp| ["#{sp.description}", sp.id]} %><br /> <% form.buttons do %><br /> <%= form.commit_button "Continue" %><br /> <% end %><br /><% end %><br /></code></pre><br /><br /><span style="font-weight:bold;">Step 2:<br />Routes!<br /></span>We needed to add some custom routes to the users block to let Devise know what was going on:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code> devise_for :users do<br /> get "/users/sign_up" => "subscriptions#new"<br /> get "/users/sign_up2" => "registrations#new"<br /> end<br /></code></pre><br /><br />The first line says use the subscriptions controller as the first stage of the sign up. That way we don't need to change any helpers and can do things in the standard Devise way. The second adds the second stage - the actual user new-ing and creation:<br /><br /><span style="font-weight:bold;">Step 3:<br />Override the Registrations Controller.<br /></span>This is where 'the good stuff' happens! Note that we don't need to override the create method. Devise has a hook in it for building the model, so we override that instead.<br />It should be noted that this only works if the model can't be saved - i.e. your validations are complete. In our case we are creating 3 models in 1: subscription, address and user. The subscription and address will get saved if the user gets saved. So, we have validations in the user requiring a subscription and address. Then validations on the address to make sure it is good. That way if the validation of the address fails the validation of the user fails and we get chucked back out the the new user form.<br />The code might say it better!<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>class RegistrationsController < Devise::RegistrationsController<br /> def new<br /> @address = Address.new<br /> begin<br /> @subscription_plan = SubscriptionPlan.find(params[:subscription][:plan_id])<br /> rescue Exception => e<br /> redirect_to users_sign_up_path and return<br /> end<br /> @user = User.new<br /> end<br /> <br /> protected<br /><br /> # Build a devise resource passing in the session. Useful to move<br /> # temporary session data to the newly created user.<br /> def build_resource(hash=nil)<br /> address_info = params[:user].delete(:address) rescue {}<br /> begin<br /> sub_info = params[:user].delete(:subscription_plan)<br /> @subscription_plan = SubscriptionPlan.find sub_info["id"]<br /> rescue Exception => e<br /> redirect_to users_sign_up_path and return<br /> end<br /> <br /> subscription = Subscription.new(:subscription_plan_id => @subscription_plan.id)<br /> <br /> @address = Address.new(address_info.merge(:country => "US"))<br /> <br /> @user = User.new(params[:user].merge(:first_name => @address.first_name, :last_name => @address.last_name))<br /> @user.subscription = subscription<br /> @user.current_shipping_address = @address<br /> end<br /><br /> def after_sign_up_path_for(resource)<br /> new_user_billing_detail_path(resource)<br /> end<br /> <br />end<br /></code></pre><br /><br />Also note that we override 'after_sign_up_path_for'. That way we move the user onto the third stage of the process - the billing info - if they sign up.<br /><br />This is still in early stages, and not yet live, but the process seems to work.Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-3255107130863381139.post-1023514691537662782010-10-14T12:26:00.003+02:002010-10-14T12:30:44.470+02:00testing Iridium gateway and active merchant with RspecRecently I rewrote the billing section of a website. The site uses Iridium as the gateway.<br />I thought people might be interested in seeing how I spec'd this:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><br /> context "iridium" do<br /> before do<br /> BillingDetail.gateway = <br /> ActiveMerchant::Billing::IridiumGateway.new(<br /> :login => "Casdasdasd",<br /> :password => "asdasdasd",<br /> :enable_3d_secure => true)<br /> end<br /> <br /> it "should work" do<br /> bd = Factory.build(:unsaved_billing_detail)<br /><br /> bd.should_receive(:ready!)<br /><br /> credit_card = Factory.build(:iridium_good_no_3ds)<br /> address = Factory.build(:iridium_good_no_3ds_address)<br /> bd.address = address<br /><br /> bd.authorize(100, credit_card)<br /> end<br /><br /> it "should fail" do<br /> bd = Factory.build(:unsaved_billing_detail)<br /><br /> bd.should_receive(:fail!)<br /><br /> credit_card = Factory.build(:iridium_card_declined)<br /> address = Factory.build(:iridium_card_declined_address)<br /> bd.address = address<br /><br /> lambda {<br /> bd.authorize(100, credit_card)<br /> }.should raise_error(TGR::GatewayError)<br /> end<br /> end<br /><br /></code></pre><br /><br />I use FactoryGirl, so I set up some factories:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><br />Factory.define :iridium_good_no_3ds, :class => ActiveMerchant::Billing::CreditCard do |cc|<br /> cc.number "4976000000003436"<br /> cc.last_name "Watson"<br /> cc.first_name "John"<br /> cc.verification_value "452"<br /> cc.month "12"<br /> cc.year "2012"<br />end<br /><br />Factory.define :iridium_good_no_3ds_address, :class => Address do |address|<br /> address.last_name "Watson"<br /> address.first_name "John"<br /> address.address_1 "32 Edward Street"<br /> address.city "Camborne"<br /> address.state "Cornwall"<br /> address.zipcode "TR14 8PA"<br /> address.country "GB"<br />end<br /><br /><br />Factory.define :iridium_card_declined, :class => ActiveMerchant::Billing::CreditCard do |cc|<br /> cc.number "4921810000009076"<br /> cc.last_name "Lewis"<br /> cc.first_name "Jack"<br /> cc.verification_value "875"<br /> cc.month "12"<br /> cc.year "2012"<br />end<br /><br />Factory.define :iridium_card_declined_address, :class => Address do |address|<br /> address.last_name "Lewis"<br /> address.first_name "Jack"<br /> address.address_1 "4 Wing Road"<br /> address.city "Leighton Buzzard"<br /> address.state "Bedfordshire"<br /> address.zipcode "LU7 0JB"<br /> address.country "GB"<br />end<br /></code></pre><br /><br />This seems to work a treat. I can do all my other tests using the bogus gateway, but to make sure I am actually providing the right info for Iridium this context, with its before block to set the right gateway, does the job nicely!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-30430247508603450522010-02-26T19:54:00.003+01:002010-02-26T20:00:51.037+01:00upgrading from ruby 1.8.6 to 1.8.7We recently updated filmamora.com to use the latest passenger (2.2.10) and Ruby Enterprise Edition 1.8.7. This was a move from 2.2.4 and 1.8.6 respectively. (We also upgraded to Nginx 0.7.4)<br /><br />So far it seems to be a very worthwhile move! I had looked into using ruby 1.9, but ran into may issues with gems.<br /><br />Here are some Munin charts.<br /><br />Mysql... this I don't even understand! What a difference!<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_iy2w4vyQwV0/S4gZtH-T6JI/AAAAAAAAAGU/AxzLQXNguXI/s1600-h/filmamora.com-mysql_queries-week.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 218px;" src="http://2.bp.blogspot.com/_iy2w4vyQwV0/S4gZtH-T6JI/AAAAAAAAAGU/AxzLQXNguXI/s320/filmamora.com-mysql_queries-week.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5442628412702451858" /></a><br /><br />Memory - also seems to be a nice drop.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/_iy2w4vyQwV0/S4gZs4k-sgI/AAAAAAAAAGM/TZqUAxSy0KQ/s1600-h/filmamora.com-memory-week.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 264px;" src="http://4.bp.blogspot.com/_iy2w4vyQwV0/S4gZs4k-sgI/AAAAAAAAAGM/TZqUAxSy0KQ/s320/filmamora.com-memory-week.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5442628408569672194" /></a><br /><br />Individual Interupts<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_iy2w4vyQwV0/S4gZsup2vSI/AAAAAAAAAGE/8ibeE6u-HEE/s1600-h/filmamora.com-irqstats-week.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 264px;" src="http://3.bp.blogspot.com/_iy2w4vyQwV0/S4gZsup2vSI/AAAAAAAAAGE/8ibeE6u-HEE/s320/filmamora.com-irqstats-week.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5442628405905767714" /></a><br /><br />Interupts<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/_iy2w4vyQwV0/S4gZsQmsFAI/AAAAAAAAAF8/iOUPFXynI1w/s1600-h/filmamora.com-interrupts-week.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 178px;" src="http://4.bp.blogspot.com/_iy2w4vyQwV0/S4gZsQmsFAI/AAAAAAAAAF8/iOUPFXynI1w/s320/filmamora.com-interrupts-week.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5442628397839422466" /></a><br /><br />The webapp itself was not changed at all. it is still using Rails 2.2.2.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-5762335444195634602009-12-20T10:22:00.002+01:002009-12-20T10:25:18.345+01:00SSL_RequirementI use the ssl_requirement plugin to let me specify what actions need to be secure.<br /><br />But - I found it limiting. I needed to:<br /><ol><br /><li>Specify what the secure domain was - we only have a certificate on the base domain, not all the subdomains</li><br /><li>Be able to easily turn it off in different environments</li><br /></ol><br />So, I forked a version off that does that. If you have the same needs (and, like me, prefer a plugin over a gem) <a href="http://bit.ly/4oyKBJ">check mine out</a>!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-70356206740876068672009-12-19T20:32:00.003+01:002009-12-19T20:46:33.748+01:00Apache + Passenger + SSL + OSXRecently I wanted to do some work on parameter passing and also how to keep parameters passed to SSL pages through redirects.<br />The first stumbling block was to get passenger and apache to play nice with SSL on OSX<br /><br />I am running Leopard (not SNOW Leopard) and here is what I did to get it to work. I couldn't find anything specific on the net about this, so I thought I'd chuck this up here. There may be nicer, better ways of doing this.<br /><br /><h2>1 - Create the cert</h2><br />Apple has a <a href="http://bit.ly/7wdxx6">page</a> on creating a cert. This all worked fine except the locations aren't right. This page must be for Tiger.<br />For Leopard apache is in /etc/apache2 (or /private/etc/apache2 depending on your installation).<br />It all seemed to work fine for me as written apart from that.<br />You end up with a ssl.key directory in your apache2 directory. You may wish to rename this domain.ssl.key if you are doing multi domain development. I am, but this is the only domain I wanted to check ssl on.<br /><br /><h2>2 - Apache and SSL</h2><br />This turned out to be easier than expected, but everything's easy when you know how!<br /><span style="font-weight:bold;">Edit /etc/apache2/httpd.conf</span><br />All I needed to do is put this line around line 40:<br /><code>Listen 80<br />Listen 443</code><br /><br />You add the Listen 443<br /><br /><h2>3 - Passenger files</h2><br />I did this by hand. As far as I know you can't do this via the prefpane,<br />I have used the PrefPane to create the vhosts file.<br /><code>sudo vi passenger_pane_vhosts/mydomain.local.vhost.conf</code><br /><br />Then add:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code><VirtualHost *:443><br /> ServerName mydomain.local<br /> ServerAlias mydomain.local es.mydomain.local en.mydomain.local<br /> DocumentRoot "/Users/smyp/development/mydomain/public"<br /> RailsEnv development<br /> <directory "/Users/smyp/development/mydomain/public"><br /> Order allow,deny<br /> Allow from all<br /> </directory><br /><br /># SSL Configuration<br />SSLEngine on<br />SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP<br />SSLOptions +FakeBasicAuth +ExportCertData +StdEnvVars +StrictRequire<br /><br />#Self Signed certificates<br /> SSLCertificateFile /private/etc/apache2/ssl.key/server.crt<br /> SSLCertificateKeyFile /private/etc/apache2/ssl.key/server.key<br /> SSLCertificateChainFile /private/etc/apache2/ssl.key/ca.crt<br /><br />SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0<br /><br /></VirtualHost><br /></code></pre><br /><br />Basically what you do is copy all the stuff from the <VirtualHost *:80> area and then add in the extra SSL config.<br />Just point the crt, key and ca.crt files to the ones you created in step 1 from the apple doc.<br /><br />That's it! You should be ready to go!<br /><br />Let me know if there are any errors in this and I'll correct them... I wasn't making notes as I went along, so this is done from looking back, so maybe I've left something out.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3255107130863381139.post-26082298780593593722009-12-16T12:05:00.003+01:002009-12-16T12:09:10.574+01:00redirect_to : pass all parametersOn <a href="http://filmamora.com">filmamora</a> we redirect people to the language version we think they want. This is done with a subdomain for the language.<br />As if that wasn't painful enough, it now seems I was stripping out additional parameters - like the parameters passed in from our mailings that triggered Google Analytics campaigns.<br />Doh!<br /><br />So this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code> def language_redirect<br /> redirect_to :subdomain => session[:language] unless RAILS_ENV == 'staging'<br /> end<br /></code></pre><br /><br />Needed to become this:<br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code> def language_redirect<br /> p2 = params.merge({:subdomain => session[:language]})<br /> redirect_to p2 unless RAILS_ENV == 'staging'<br /> end<br /></code></pre><br /><br />Such a small change - such a big difference! Marketing types appeased!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-30836003089588189342009-10-06T15:01:00.002+02:002009-10-06T15:03:48.266+02:00rotating nginx logsSince moving most of my systems over to plain Ubuntu on EC2 (as opposed to using ec2OnRails) I've needed to figure out certain things - like rotating the nginx logs (using Nginx instead of apache).<br /><br />Simply create a file in <code>/etc/logrotate.d/nginx</code> that looks like this<br /><br /><pre>/opt/nginx/logs/*.log {<br /> daily<br /> missingok<br /> rotate 30<br /> compress<br /> delaycompress<br /> sharedscripts<br /> postrotate<br /> /etc/init.d/nginx restart<br /> endscript<br />}</pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3255107130863381139.post-14104677721594581242009-07-24T15:05:00.002+02:002009-07-24T15:09:09.586+02:00move from ec2onrailsRecently the performance on FilmAmora.com has been plummeting. And I had no idea why. Often apache was timing out waiting for a mongrel. Very little development has happened recently on FilmAmora, so I was stumped.<br />So - like a good developer - I took this as a chance to make a drastic move from ec2onrails machines to straight ubuntu, with everything else installed by hand.<br /><br />It has made a HUGE difference. So much so that I could actually drop a machine from the configuration and still have better response times.<br /><br />I followed these instructions to get started: <a href="http://github.com/jnstq/rails-nginx-passenger-ubuntu/tree/master">rails-nginx-passenger-ubuntu</a>. I also switched to nginx and ruby enterprise edition - just to make following the instructions easier.<br /><br />So far so good. It took a couple hours and presto a leaner meaner installation. My NewRelic Apdex score went from .5 to .9.Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3255107130863381139.post-91082852040432188992009-07-17T12:10:00.001+02:002009-07-17T12:10:38.761+02:00expiring action_cacheI was having a bugger of a time expiring my cache. The problem was the cache_path.<br />I have index actions that take filters. So, I created a cache_path for my action caching:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>caches_action :index, :layout => false, :cache_path => Proc.new { |c| "#{c.params[:league_id]}/index/#{c.params.values.sort.to_s}" }<br /></code></pre><br /><br />Which means that all my caches end up in a nice tree of 'views/[league name]'. But, how to expire a whole part of the cache tree when there could be dozens of nodes as people could be searching on last names too.<br /><br />What I found was that expire_action was NOT the way to go. expire_fragment lets you pass in a regular expression. Now in my sweeper I have this:<br /><br /><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code> def expire_cache_for(player)<br /> expire_fragment(%r{views/#{player.league.to_param}.*})<br /> end<br /></code></pre><br /><br />Which will expire all the cached code for the player's league.Unknownnoreply@blogger.com0