Some developers saw their Twitter bots begin replying multiple times to every message yesterday. We apologize for that and are working to prevent it from happening again in the future.

For those interested in the technical details of the issue, here’s what happened.

Our application has an exemption from Twitter’s normal API rate limits. Where normally users are only able to make 150 requests per hour to the API, each of our servers are able to make 20,000 requests per hour, and requests made from our servers to a Twitter ID do not count toward that Twitter ID’s normal 150 per hour rate limit. This exemption allows us to respond to messages within moments of them being sent to your Twitter account as well as allowing us to manage a large number of Twitter accounts.

Because the number of requests we need to make to process your messages varies depending on how many messages are sent to your bot and how many messages you send from your bot to other people, we constantly re-evaluate the API rate limit status of each server. This ensures that we always have enough remaining connections to reply to messages. Our system checks with Twitter to see how many API calls are remaining and how long before the rate limit count is reset. We then increase or decrease the pace of our API calls to fit within the remaining limit.

Our normal workflow is to check the rate limit, adjust our API pace, then connect to every Twitter account we’re monitoring, download new messages, and post replies. Then we start the process again.

Unfortunately an issue with Twitter’s API exposed a race condition that we hadn’t anticipated. For those unfamiliar with race conditions, they occur when two processes attempt to do the same thing at the same time or when they both attempt to modify the same data. Each process is unaware that there’s another process doing the same thing, resulting in inconsistent and unpredictable results.

Yesterday, Twitter increased our rate limit unintentionally. Instead of each server making 20,000 requests per hour, we suddenly were allowed to make 20,000 requests per second. Our code that constantly adjusts the pace of our API calls performed it’s magic and sped up our system. Instead of processing your messages every few seconds, we started processing them several dozen times per second. Often, in an attempt to keep up, our system would attempt to process the same message multiple times simultaneously, causing a race condition. This resulted in your replies being posted several times. Sometimes another race condition occurred, causing us to re-process the same messages we’d already downloaded earlier in the day. These two issues combined to post your replies several times, sometimes over the course of several hours.

Some users saw this compounded by an unrelated issue. One of our servers went offline and when it came back, the Twitter process did not properly restart. This prevented us from processing messages for some Twitter accounts. When we restarted it, some of these accounts had a backlog of messages to process. The restart happened after the race condition issue appeared, resulting in this backlog of messages often getting processed multiple times. So instead of just a couple of messages getting multiple replies, many messages were replied to multiple times.

We’re taking several steps to prevent this from re-occurring. We’ve added some sanity checks rate limit logic to ensure that we don’t hit the Twitter API at an unreasonable pace, even if Twitter tells us it’s okay to do so. We’re in the process of adding safeguards that will guarantee that multiple processes don’t attempt to fetch and reply to the same account at the same time. To prevent the race condition from happening again while we work to implement these safeguards, we’ve placed an artificial cap on the volume of our API requests. This may cause your Twitter bots to respond more slowly than normal until we finish fixing the issue. We anticipate releasing the fix and lifting our self-imposed cap within a few days.

You may find your bots go offline for a bit while we upgrade some systems here. We’re putting in place a new system that we hope will offer even more stability and reliability for your hosted bots. Thanks for your patience while we get the new stuff in place.

Update 9/29 at 1am - Upgrades complete

Last night we launched our new site along with a bit of a change in direction for IMified. We started out as a tool (IM bot) that made it easy to connect your instant messenger to popular web applications. We quickly realized that there are a lot of developers out there looking for a simple way to build IM bots and connect them to their applications. V2 of IMified aims to solve that problem by offering a web-based platform that allows developers to write their IM application once in any language, host it on any web server, and point it to any number of screen names on public IM networks like Gtalk™, AIM™, MSN™, and Yahoo™.

What’s new? A lot. Our new site introduces the IMified developer platform for quickly and easily launching IM applications (bots). Our FREE developer platform includes:

• Up to three custom Gtalk/Jabber screen names on our bot.im domain
• The ability to connect your bot to public networks. (Gtalk™, AIM™, MSN™, Yahoo™)
• An interactive real-time debugger for testing your bots
• Simple REST API for sending IM messages/alerts and polling for IM status (presence)
• Live real-time usage statistics on your applications
• Free developer support
• Publicize your bot in the App Directory (soon)

** The platform is still in beta so please send feedback

What will pricing look like when the beta is over? Pricing is still being determined but our goal is offer an easy and affordable way to create IM bots and we intend to always offer a free and low cost hosting option.

Ok, so what’s happening to the existing IMified bots? Nothing, they’re still there. You can find our old home page here and you can still add/edit widgets to your IMified menu. We’re working on moving all of our widgets into the new App Directory (coming soon). Each widget will have its own screen name and in turn offer more/better functionality.

We’re looking forward to seeing what you create on the new platform. Feedback is welcome and is appreciated.

We are aware that new bots are not coming online immediately after signing up on the site. We made an unexpected appearance on the Digg home page yesterday into today and are working to get your new bots online as quickly as possible. Under normal load they’d be online right away. Please look here for an announcement when things are back to normal and thank you for trying us out!

UPDATE: We’ve turned a number of bots on. If your bot is not one of them, hang on, we’re working on it.

UPDATE 2: We’re completely back.

Update: Google has been very helpful and resolved the issue. Those with hosted bot.im bots may now talk to them via Gtalk and any other jabber network.

We recently sent out a slew of beta invites to our bot hosting platform and have opened up access to anyone who wants to try it out. A big thanks to everyone who’s waited patiently and have setup a bot!

In anticipation of opening up the platform we decided to host all new jabber/gtalk bots on a new domain, bot.im, as opposed to the imified.com domain. This allows us to seperate the core IMified bot traffic (which can be hefty) from our hosted bots and also offers developers a more generic domain to run your bot on. Plus, we just think bot.im is a pretty cool domain!

Now to address the title of this post. You may have noticed that you cannot access your yourbot@bot.im screen name from your google talk account.  We figured out the issue after quite a bit of hair pulling and it appears what happened is someone, at some point, registered our new bot.im domain with Google Apps for Your Domain.  Obviously this person never confirmed they own the domain, being that we do, but regardless Google still “locks out” anyone from administering the domain in Gapps. Now, we host our own jabber servers but Google will route any jabber traffic to and from a Google Apps domain through their jabber servers. So when you try to talk to your bot from gtalk, google is sending your message to their jabber servers as opposed to ours.

As crazy as the above sounds, its true, and not something I’d expect from Google. See this Google help page for the strange truth. You’d think that a domain would not be locked until someone proved that they actually own the domain. We’ve put in a request to Google to please release our domain and not allow anyone else to set up a Gapps account with it. As soon as it’s resolved we’ll send out an email to everyone with a beta bot.

Thanks for sticking with us!

IMified is pleased to announce the launch of our hosted services available through the IMified platform. Now anyone can create an IM bot just as easily as creating a web application.

We’re convinced that the ubiquity and ease of use of IM makes it important for everyone to IMify their web apps. So we’ve created a platform that makes it fast and simple to create and deploy IM integration that runs over multiple networks. A simple RESTful API allows you to add IM capabilities to your existing application using the programing tools, techniques, and server components you’re already using for your web site.

You don’t need to build a web server to create a great web app. You shouldn’t have to build an IM server to create a great IM app.

We want everyone to be able to add IM to their sites, so we’ve created a pricing structure that allows anyone to get involved. You can create widgets that run in the IMified bot for free or get your own Jabber/Gtalk ID for as little as $10 a month. For more serious usage, we offer upgraded services that can handle the needs of the most demanding bot.

Want to IMify your app? Check out our developer pages for details on the API, our pricing, and to sign up. The service is in a limited beta right now, but if you leave your contact details, we’ll send you an invite as slots open up.

For those of you who have taken advantage of the now deprecated widget API , your widgets will continue to function as normal. We’ll send you details on how you can take advantage of the new API as we get closer to opening up the platform, however you may request an invite to the beta if you’d like a sneak peak.

We think Pownce is really cool. So cool in fact that we did some digging around in their (unofficial) API to see if we could add some sort of IM support to the uber anticipated messaging app. Thanks to guys like Kyle Hays, Mike Taylor, and Clint Ecker, who discovered the hidden api and documented its login scheme, we were able to integrate most Pownce features into a new IMified widget. To add the widget just select the “My Account” option in your IMified menu and click the link provided to access the widget directory.

The Pownce widget will let you post messages, links, events, and sends IM notifications of new Pownce messages. Notifications are checked for every 120 seconds. To enable notifications, select the “Toggle Notifications” option in your Pownce menu.

As stated above, this widget was built on the “Unofficial” Pownce api. As far as we can tell the team over at Pownce isn’t restricting access to it, however they’re not encouraging developers to use it. A recent blog entry on the Pownce site says they’re working on an official API, but we just couldn’t wait to release the first Pownce addon (that we know of).

If you’ve got any suggestions for improving the widget, or if it’s just not working for you, let us know in the forums.

BTW, we have some Pownce invites. Leave a comment if you want one. We’re “IMified” on Pownce if your looking for another friend : )

Sorry for the delay on this one. Just added a new Blogger widget that will allow you to post to your blogger blog using your google account. The widget makes use of the new Gdata api so your login credentials are not stored on our servers. If you are using the prior blogger widget, please delete it from your menu and add the new one.

The posts here have been a bit slow mostly because we’ve been working really hard to publish the first version of the IMified API. With the new API also comes the introduction of what we call IM widgets. IM widgets are services that can be interacted with and added to your IMified menu and can now be developed by anyone, in any language.  Check out the API docs and give it a whirl. We can’t wait to see what kind of stuff you guys come up with. We feel like an API was the next logical step for IMified. It’s a lot of fun seeking out and integrated services, but there’s just so much we can do in a day. Our hope is that application owners will embrace our api and build widgets that their users can interact with on IMified. So, if you’d like to see your favorite app on IMified, send them a link to our API and tell them to get crackin.

Our first public widget written on the api was created by the awesome url shortening site urlTea.com. We’ve also added a new Jaiku widget to the gallery that lets you post updates to your Jaiku account. We’re working on building in Jaiku updates as well, so you can receive updates from your contacts as well as comments from any of your Jaiku posts.

Thanks to all for your patience while we continue to do our best to scale our bots and squash some of the bugs in our widgets. Please visit the forums for help creating your widgets or to just say hi. You can also always send us an email at imified@imified.com.. we’d love to hear from you.

Over the weekend we added Twitter as a new IMified service. We definitely feel their pain trying to keep an IM bot up and running. We’ve had our own issues in case you haven’t noticed ; ) So when the twitter bot is down feel free to get your twit on over here. For now, all you can do is post status updates but we may build in some other api features if you guys want them. Let us know.

Btw, our twitter id is, you guessed it, imified. We’ll be posting dev news and service updates so feel free to add us, we can use the friends.