I’ve been working on threading a Rails application lately and after reading headlines like ‘Rails is thread safe’ I figured how hard could it be. My first discovery was that when people talk about Rails and threads there are two different types of threading in Rails.

Multiple request threading – This is where Rails itself threads among different requests to your web server and allows ActiveRecord to behave properly without having to keep a copy of Rails in memory for each request.
Intra-request threading – This is where you have 1 request to your web server and inside the action you want to create multiple threads that run concurrently.

I’ll be talking about Intra-request threading. In particular, I want my threads to execute some code, read and write to the database, and play nicely with each other. My first attempt was to use the Ruby Thread method. This seemed to work somewhat until I started seeing strange errors coming in from MySQL. The problems seemed to occur randomly and what I determined was that the threads were trying to write to the database at the same time which ended up causing some collisions of sorts resulting in ‘lock wait timeout exceeded’ errors.

After considerable Googling, I found numerous posts about setting:

ActiveRecord::Base.allow_concurrency = true

The problem with that is that this is deprecated in the newest version of Rails in favor of connection pooling.

The short answer: Don’t use the Ruby Thread method within Rails when doing anything with ActiveRecord.

Here I am yet again at a standstill because the Rails gems have mostly all migrated to Github and the site is down. This is the second or third time this year that Github has messed up and caused me to stop dead in my tracks because I couldn’t access the gems I needed. I can’t even remember the last time my subversion repository went down. This is exactly the reason I thought the Rails community was crazy for mass migrating to Github. Its just not a good idea to have 1 centralized location responsible for all your gems. Especially when they go down all the time. What we need is a company with experience maintaining quality uptime to manage this site in the short term and in the long term we need a more distributed mirror type system to roll-over when Github dies again.

Github down again

I’ve just finished adding AOL support to the Rails contacts gem.

Until this is officially added to the contacts gem release, you’ll need to create a RAILS_ROOT/lib/contacts/aol.rb file with the following contents:

class Hash
def to_query_string
u = ERB::Util.method(:u)
map { |k, v|
u.call(k) + "=" + u.call(v)
}.join("&")
end
end

class Contacts
require ‘hpricot’
class Aol < Base
URL = "http://www.aol.com/"
LOGIN_URL = "https://my.screenname.aol.com/_cqr/login/login.psp"
LOGIN_REFERER_URL = "http://webmail.aol.com/"
LOGIN_REFERER_PATH = "sitedomain=sns.webmail.aol.com&lang=en&locale=us&authLev=0&siteState=ver%3a4|rt%3aSTANDARD|ac%3aWS|at%3aSNS|ld%3awebmail.aol.com|uv%3aAOL|lc%3aen-us|mt%3aAOL|snt%3aScreenName&offerId=mail-second-en-us&seamless=novl&xchk=false"
# http://webmail.aol.com/28878/aim-2/en-us/Suite.aspx
CONTACT_LIST_URL = "http://webmail.aol.com/28878/aim-2/en-us/Lite/ContactList.aspx?folder=Inbox&showUserFolders=False"
CONTACT_LIST_CSV_URL = "http://webmail.aol.com/28878/aim-2/en-us/Lite/ABExport.aspx?command=all"
PROTOCOL_ERROR = "AOL has changed its protocols, please upgrade this library first. If that does not work, dive into the code and submit a patch at http://github.com/cardmagic/contacts"

def real_connect

postdata = {
"loginId" => login,
"password" => password,
"rememberMe" => "on",
"_sns_fg_color_" => "",
"_sns_err_color_" => "",
"_sns_link_color_" => "",
"_sns_width_" => "",
"_sns_height_" => "",
"offerId" => "mail-second-en-us",
"_sns_bg_color_" => "",
"sitedomain" => "sns.webmail.aol.com",
"regPromoCode" => "",
"mcState" => "initialized",
"uitype" => "std",
"siteId" => "",
"lang" => "en",
"locale" => "us",
"authLev" => "0",
"siteState" => "ver%3A4%7Crt%3ASTANDARD%7Cac%3AWS%7Cat%3ASNS%7Cld%3Awebmail.aol.com%7Cuv%3AAOL%7Clc%3Aen-us%7Cmt%3AAOL%7Csnt%3AScreenName",
"isSiteStateEncoded" => "true",
"use_aam" => "0",
"seamless" => "novl",
"aolsubmit" => CGI.escape("Sign In"),
"idType" => "SN",
"usrd" => "4222992",
"doSSL" => "",
"redirType" => "",
"xchk" => "false"
}

# Get this cookie and stick it in the form to confirm to Aol that your cookies work
data, resp, cookies, forward = get(URL)
postdata["stips"] = cookie_hash_from_string(cookies)["stips"]
postdata["tst"] = cookie_hash_from_string(cookies)["tst"]

data, resp, cookies, forward, old_url = get(LOGIN_REFERER_URL, cookies) + [URL]
until forward.nil?
data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
end

doc = Hpricot(data)
(doc/:input).each do |input|
postdata["usrd"] = input.attributes["value"] if input.attributes["name"] == "usrd"
end
# parse data for <input name="usrd" value="2726212" type="hidden"/> and add it to the postdata

postdata["SNS_SC"] = cookie_hash_from_string(cookies)["SNS_SC"]
postdata["SNS_LDC"] = cookie_hash_from_string(cookies)["SNS_LDC"]
postdata["LTState"] = cookie_hash_from_string(cookies)["LTState"]

data, resp, cookies, forward, old_url = post(LOGIN_URL, postdata.to_query_string, cookies, LOGIN_REFERER_URL) + [LOGIN_REFERER_URL]

until forward.nil?
data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
end

if data.index("Invalid Screen Name or Password.")
raise AuthenticationError, "Username and password do not match"
elsif data.index("Required field must not be blank")
raise AuthenticationError, "Login and password must not be blank"
elsif data.index("errormsg_0_logincaptcha")
raise AuthenticationError, "Captcha error"
elsif data.index("Invalid request")
raise ConnectionError, PROTOCOL_ERROR
elsif cookies == ""
raise ConnectionError, PROTOCOL_ERROR
end

@cookies = cookies
end

def contacts
postdata = {
"file" => ‘contacts’,
"fileType" => ‘csv’
}

return @contacts if @contacts
if connected?
data, resp, @cookies, forward, old_url = get(CONTACT_LIST_URL, @cookies, CONTACT_LIST_URL) + [CONTACT_LIST_URL]

until forward.nil?
data, resp, cookies, forward, old_url = get(forward, cookies, old_url) + [forward]
end

if resp.code_type != Net::HTTPOK
raise ConnectionError, self.class.const_get(:PROTOCOL_ERROR)
end

# parse data and grab <input name="user" value="8QzMPIAKs2" type="hidden"/>
doc = Hpricot(data)
(doc/:input).each do |input|
postdata["user"] = input.attributes["value"] if input.attributes["name"] == "user"
end

data, resp, cookies, forward, old_url = get(CONTACT_LIST_CSV_URL, @cookies, CONTACT_LIST_URL) + [CONTACT_LIST_URL]

parse data
end
end
private

def parse(data, options={})
data = CSV.parse(data)
col_names = data.shift
@contacts = data.map do |person|
["#{person[0]} #{person[1]}", person[4]] unless person[4].empty?
end.compact
end
end

TYPES[:aol] = Aol
end

I’ve submitted the source to the authors so it should be added in the next few weeks.

UPDATE: My changes have been merged into the trunk.

I’ve been writing a lot about how great Bing is and made the switch to use Bing regularly but here I am in day 1 with using Bing full time and have run into problems. I am working on a Rails project doing some http requests and screen scraping. Basically, I’m extending the contacts gem to include AOL support. I wanted to see a quick reference to refresh my memory on the get / post syntax in Rails. I search Bing for “rails get http” and the first result is a link to rubyonrails.org. How useless. I search the entire first page and find absolutely nothing related to what I’m looking for. I’m beginning to second guess my previous whole-hearted support of Bing. Its good for general searches but it seems program related searches are crap with Bing. I do the same search in Google and the first result is what I wanted. Just a quick reference of someone using the Ruby get method. I tried a few other searches like ‘rails contacts gem’ and Bing doesn’t really give me much to work with in the results. I did have to dig down to the 8th result with Google but the Rails contacts gem project page is at least on the first page of the search results. Google wins when it comes to programming related searches in my opinion. Time for me to switch back to Google for the time being.

I’ve made the switch to Bing as my default browser in Firefox. It doesn’t come as an option with Firefox but there is an add-on you can easily install to add support for Bing in the Firefox search bar. Finally, some competition in search and finally Microsoft releases a quality web application. You can get the add-on here: https://addons.mozilla.org/en-US/firefox/addon/10434

I’ve long been anti-Microsoft migrating first to Linux and then to Mac OSX. I’ve been using OpenOffice since the days it was first released (remember Star Office?). I used Mozilla long before Firefox every came out and even stopped supporting some older versions of IE with Onomojo. Needless to say, Microsoft would have to do something extremely well to catch my eye and even more so to actually get me to use it.

I have a confession to make. I’ve been using Microsoft’s Bing search engine more and more lately. I feel my results are more relevant and less spammy. It seems these days 5 out of the 10 search results on Google are simply screen scrapers reposting content and dumping Google Adwords all over the place. Of course Google is going to rank those higher simply in the name of profits. They of obviously will never admit to that though but think about it. If they generally rank sites with Adwords on them higher, they can make a significant more amount of money per day. Its hard to believe any corporation would ignore that fact. Certainly, a publicly traded company is at the mercy of shareholders who are only interested in the pursuit of more profits. With that being said, Google still has to maintain credibility in its search results so the same game applies but I simply refuse to accept the idea that Google doesn’t inflate rankings of sites that use Adwords onsite. Just do a few random searches and you’ll see what I mean.

I’ve been finding that Microsoft’s Bing search engine is pulling more relevant results and less junk results than Google. I don’t have any data to back that up. Its just my general perception of the results. One thing I really like is Bing’s image search. It does an Ajax call to keep the page scrolling down so you never have to click on the next page. Hover over an image and it magnifies it some with details. All in all, Microsoft is onto something good with their Bing search engine. I think its going to be a major competitor within the next year. I’ve already noticed that I go to Bing first for some types of searches.

People are trendy. They like the latest trends and naturally migrate towards trendy sites. Bing will win more and more search traffic if Google doesn’t up their game. Google long ago stopped actually innovating and instead went into operational mode since there’s no real competition out there. They’ve just been trying to position themselves to maintain that dominance without actually bringing anything new to the table. Bing is coming out swinging. Its only been a few months since it went live and its already making converts out of long time anti-Microsoft people like myself. In my opinion, that says a lot both about the technology behind Bing and the lack of innovation from Google. Take it for what its worth.

I’m happy to announce a Flash addition to our Onomojo homepage. The artwork was inspired by an artist here on the North Shore of Oahu named Heather Brown. The actual artist for the Flash animation and the latest addition to the Onomojo team is Rhys Lynn. Here’s a preview of the new splash page and the new homepage. The homepage just changed backgrounds from the green dots that were there before but it looks a million times better with the blue background. Let me know what you think.

I have numerous SEO clients and one thing that I’ve noticed in the past few months is a drastic improvement in Microsoft’s Bing search engine’s quickness to respond to on-site content changes. All my clients are performing extremely well on Microsoft’s Bing search engine first and then the others follow. Yahoo and Google seem on par with each other as far as quickness to update and in rankings in general. Yahoo does seem to occasionally throw in huge random ups and downs but on the most part its in sync with Google’s rankings for most of my clients. Bing’s results however all outperform the other search engines in terms of quickness to respond to content updates as well as its ability to maintain high rankings for new content over time.

I’m not sure what Microsoft did to their search engine when they rebranded it as Bing but it does appear to behave differently as far as rankings for my clients. Despite the low amount of traffic it sends it is positive news either way. Now if only they offered some quality free tools like Google’s Webmaster Tools and Google Analytics it might start making some converts.

What I would personally like to see is for a top search engine to publish a detailed description for how to get them to view your site positively. They all keep it behind a black box and let the magic behind SEO consultants do the research but why? Why not just come out and explicitly say here is how to get your site ranked well. Here is what we don’t like. Here is what we do like. Then provide useful tools for optimizing your site to make it better. Offer a free set of SEO tools direct from the search engine. Then have a paid version that goes a lot deeper. I’m sure I’d have all my clients on the paid version in no time. Its not paying for rankings, its paying for the tools to get the site ranked better.

Its a whole new twist on SEO that who ever takes advantage of first might just become the next king of search.

Note: Microsoft didn’t pay me to write this but they should. They need all the positive publicity they can get.

The Wall Street Journal is reporting on the FTC’s recent decision to fine bloggers for not explicitly stating any compensation received for writing a blog. The FTC will now supposedly fine bloggers for writing reviews of products or services without specifying that they’ve been compensated for it. While the FTC has much bigger problems they should be dealing with like metered cell phone usage, misleading multi-year cell contracts, and software patents in general they are now focused in a direction they have absolutely no jurisdiction over. Will we now have federal blog readers employed with US tax dollars to just read blogs all day and make sure no one is writing reviews without specifying compensation for the review?

The whole logistics of enforcement of such fines is laughable in my opinion. First, what if I host my site in another country? Will these fines still apply? Second, a lot of blog sites have multiple authors with no realistic way for a site owner to ensure every writer is following the rules. Who will be responsible if a writer blogs against the rules? Finally, what will the FTC do if my site in another country and I am marketing specifically for US customer? Are they planning a great US firewall like China so they can excerpt more control over our online commerce and enforce penalties like blocking overseas sites?

Why the FTC even care? Marketing companies have been hiring bloggers to write reviews for years on sites like Pay per Post. My guess is the FTC is really less interested in protecting consumers and instead focused on brainstorming ways to help potentially generate more revenue for the US government and themselves. Its a laughable waste of tax dollars in my opinion.

NOTICE ON COMPENSATION: I received no compensation from the Wall Street Journal or the FTC for writing this blog.

Here is Micah Friedline with another screencast. This time it’s about The World’s Oyster, a self-updating address book. Stay in touch with friends, business contacts, and more with the easy to use address book that keeps itself updated as you move.