On Saturday, January 29th our network operations team will be performing network upgrades from the hours of 1am est. – 5am est.

All IM screen names will be offline during this time and SMS messages will not be routed to bots.

We expect all service to be restored around 5am on the 29th.

Some of you may have heard that Twitter is dropping support for one of the authentication methods that their API supports in favor of a new, more secure authentication system by June, 2010. IMified has always used the more secure OAuth authentication method to connect Twitter to your bot’s,  so there won’t be an issues when the change is in place.

Today we hit a bit of milestone. We’re now hosting over 10k active IM, Twitter, and SMS bots! This year in particular we’ve nearly doubled the amount of apps we’re hosting. If you are one of the 10k apps doing something cool on the platform, please let us know, we’d love to hear about it.

It’s also been nearly a year since IMified was acquired by Voxeo. Since then, IMified has been integrated with Voxeo’s hosted IVR platform as well as Tropo. We continue to work on finalizing our new bot hosting platform and anticipate opening things up for production use sometime this summer.

Over at the Voxeo Developer’s Corner blog, I’ve posted an article and sample code that shows how to add reporting to your IM bots with Google Analytics.

Using [Google's] mobile support, you can now add Google Analytics to your IM bots. Any time someone sends a message to your bot, the interaction will be sent to Google Analytics and will show up in your reports alongside your web traffic.

Apparently our account with our DNS provider has lapsed. We apologize for the downtime. DNS service has been restored and IMified.com as well as bot.im bots should be resolving soon.

Update 10:34am est: All IMified sites and bots should be resolving properly

Last month we announced the ability to “SMSify” your applications by providing you with a free SMS enabled phone number to connect to your IMified applications. We had a limited number of phone numbers available and quickly ran out. Today, we have added a bunch of new numbers to the pool and re-enabled the sms tab in the bot settings page. Grab one while they’re still available. Check out our previous post on the addition of SMS to IMified for more info.

Join us on October 7th for a free webinar when we will be teaming up with Voxeo’s VoiceObjects team to talk about building multi-channel applications that work across IM, SMS, twitter, the web, and voice.

Topic: IMify your voice application! Exploring new ways of customer interaction with Voxeo VoiceObjects and IMified

Date: Wednesday, October 7th, 2009
Time: 8am PDT, 11am EDT, 5pm CEST

Speakers:
Dave Hoff, Senior Engineer, Voxeo Tobias Goebel, Sr. Presales Consultant, Voxeo Germany

Abstract: With the acquisition of IMified and VoiceObjects, Voxeo has demonstrated its clear commitment to be the number 1 provider of Unified Self-Service™: the idea that one and the same service definition can drive dialogs in any modality available to end customers, be it voice, video, SMS, USSD, IM, Web, or even social networks.

This jam session introduces IMified, Voxeo’s hosted platform for Instant Messaging bots, and explains how VoiceObjects can be utilized to build dialogs that extend the scope of customer interaction to Instant Messaging networks such as Yahoo, Skype, AOL, MSN, or Google Talk, but also other text-based channels such as SMS or Twitter. Learn how to IMify your existing VoiceObjects application with minimal developer effort but maximum customer impact.

If you can’t attend on Wednesday, we will post a link to an archive of the webinar for viewing later. Hope to see you Wednesday!

Some developers saw their Twitter bots begin replying multiple times to every message yesterday. We apologize for that and are working to prevent it from happening again in the future.

For those interested in the technical details of the issue, here’s what happened.

Our application has an exemption from Twitter’s normal API rate limits. Where normally users are only able to make 150 requests per hour to the API, each of our servers are able to make 20,000 requests per hour, and requests made from our servers to a Twitter ID do not count toward that Twitter ID’s normal 150 per hour rate limit. This exemption allows us to respond to messages within moments of them being sent to your Twitter account as well as allowing us to manage a large number of Twitter accounts.

Because the number of requests we need to make to process your messages varies depending on how many messages are sent to your bot and how many messages you send from your bot to other people, we constantly re-evaluate the API rate limit status of each server. This ensures that we always have enough remaining connections to reply to messages. Our system checks with Twitter to see how many API calls are remaining and how long before the rate limit count is reset. We then increase or decrease the pace of our API calls to fit within the remaining limit.

Our normal workflow is to check the rate limit, adjust our API pace, then connect to every Twitter account we’re monitoring, download new messages, and post replies. Then we start the process again.

Unfortunately an issue with Twitter’s API exposed a race condition that we hadn’t anticipated. For those unfamiliar with race conditions, they occur when two processes attempt to do the same thing at the same time or when they both attempt to modify the same data. Each process is unaware that there’s another process doing the same thing, resulting in inconsistent and unpredictable results.

Yesterday, Twitter increased our rate limit unintentionally. Instead of each server making 20,000 requests per hour, we suddenly were allowed to make 20,000 requests per second. Our code that constantly adjusts the pace of our API calls performed it’s magic and sped up our system. Instead of processing your messages every few seconds, we started processing them several dozen times per second. Often, in an attempt to keep up, our system would attempt to process the same message multiple times simultaneously, causing a race condition. This resulted in your replies being posted several times. Sometimes another race condition occurred, causing us to re-process the same messages we’d already downloaded earlier in the day. These two issues combined to post your replies several times, sometimes over the course of several hours.

Some users saw this compounded by an unrelated issue. One of our servers went offline and when it came back, the Twitter process did not properly restart. This prevented us from processing messages for some Twitter accounts. When we restarted it, some of these accounts had a backlog of messages to process. The restart happened after the race condition issue appeared, resulting in this backlog of messages often getting processed multiple times. So instead of just a couple of messages getting multiple replies, many messages were replied to multiple times.

We’re taking several steps to prevent this from re-occurring. We’ve added some sanity checks rate limit logic to ensure that we don’t hit the Twitter API at an unreasonable pace, even if Twitter tells us it’s okay to do so. We’re in the process of adding safeguards that will guarantee that multiple processes don’t attempt to fetch and reply to the same account at the same time. To prevent the race condition from happening again while we work to implement these safeguards, we’ve placed an artificial cap on the volume of our API requests. This may cause your Twitter bots to respond more slowly than normal until we finish fixing the issue. We anticipate releasing the fix and lifting our self-imposed cap within a few days.

When we started IMified, we had some ideas for a couple of simple apps that ran over IM. It turned out that compared to developing a web app, sending and receiving instant messages was complicated and required understanding of all sorts of arcane rules and techniques. We wished for an easy way to send and receive instant messages. So we built a service that allowed anyone that can build a web app to plug into IM networks.

Ever since then, one of the most requested features has been SMS support. Heck, we’ve found ourselves wishing there was an easy way to send and receive text messages. Compared to delivering a web app,  sending and receiving text messages is complicated and requires understanding of all sorts of arcane rules and techniques.

Hey, that sounds familiar.

Starting today, you don’t need to understand short codes, SMS gateways, routing, DIDs or the silliness that different carriers inject into the whole process. In fact, you don’t even need to write any new code. Your existing IMified app can work over SMS.

Adding SMS to your app couldn’t be simpler. Just log into your account and select your app. Under Network Settings, click the SMS tab. Click activate and we’ll assign you a phone number.

Any text messages that get sent to that number will be delivered to your app just like an instant message. There will be a userkey, a message, and all the goodies you already know and love. Your app’s response will be sent as a return text message to the sender. You can send out text messages by sending to the user’s userkey.

If you want to respond differently to SMS users, you can do so by checking the network field in the message to your bot. Just like we already tell you if the message came from AIM, Jabber, Twitter or any of our other networks, we’ll also tell you if it’s coming via SMS.

IMified will be offline tonight, 8/17, starting at 11pm est for approximately 2 hours. We will be migrating IMified from its current DC to its new home at Voxeo. Service should resume as normal after we flip the switch.

We have created a system status blog at http://status.imified.com. Please check there for updates.