The result is called JobU and can be found here: http://renaudbourassa.com/projects/jobu/.

We won 4th place (when really, we should have won at least 3rd). The first place went to Docuvine, a really interesting web application that mimics Google Wave. Here are some picture of the event.

For more pictures of the event, see Rasmus Lerdorf or Jamie Lockwood flickr photostream.

And also, the Warriors defeated the Windsor Lancers 49-0. Let’s go Warriors!

The Classic Way

First, lets look at Twitter as it is right now. Yes, Twitter is a social application and as such, each of the network’s nodes, each user, is connected to a certain number of other nodes. The apparent structure of the service is thus the one of a peer-to-peer network or a decentralized network. However, from a technology point of view, this is not the case. Twitter is a fully centralized network.

To do whatever action you want to do, you have to go through the company’s servers. Every single request, whether it is through the Twitter public API or directly on the website, has to be processed by Twitter. All the clients do is parse the server’s response, all the hard work is done on Twitter’s side.

Why is this bad? Because Twitter’s users check often for new messages since they post a lot of messages. Also, Twitter’s open API led to the creation of many applications based on the service, the worst for the company being clients and automated services. First, a lot of clients, to stay up to date, will make frequent requests to the servers to see if something new was posted. This wouldn’t be too bad, for example, with a blog service where people only post once in a while, because the servers could use a cache system to reduce unnecessary processing. However, with its thousand of new tweets per minute (and growing), a caching system is hardly useful for Twitter. The second problem is automated applications such as Wordpress plugins or blog update services. These service also make a lot of requests, every time a blog page is loaded in the case of Wordpress and at a certain rate for the update services. Again, each request sent by the service has to be processed on Twitter’s servers, thus requiring a lot of processing power without the user even seeing the little Twitter bird.

The Efficient Way

So how can we solve this problem? Well, since Twitter is, by its social nature, decentralized, why not try to decentralize the system? This is what peer-to-peer applications such as BitTorrent and Skype have been doing for a while, and it seems to work well. Starting with the second generation of peer-to-peer networks, decentralization became a key idea. This culminated with the integration of Distributed Hash Tables (DHT) in peer-to-peer protocols and clients. From Wikipedia:

Distributed hash tables (DHTs) are a class of decentralized distributed systems that provide a lookup service similar to a hash table: (key, value) pairs are stored in the DHT, and any participating node can efficiently retrieve the value associated with a given key. Responsibility for maintaining the mapping from keys to values is distributed among the nodes, in such a way that a change in the set of participants causes a minimal amount of disruption.

So how does this apply to Twitter? Well, with its minimalist structure, Twitter is the perfect candidate for a DHT system. DHTs use key value pairs for data management and Twitter is just that, a key value system. The key is your username, and the value your tweets. What makes this an even better solution, is the simplicity of the service. Compared with Facebook activity or blog posts, tweets doesn’t include complex data such as images or text formatting. It is all about text, and small pieces of text. This not only makes storing user’s data easy, but it also requires less space. A simple XML file of an average size could be use to store a user’s full history of tweets and a list of all of his followers/following.

The idea of DHTwitter is to create an additional layer of abstraction, this time between the Twitter API and the client or application used by the end user. This API would include the overlay network and the keyspace partitioning specifications needed to establish the DHT. Once this is done, the API would redirect requests to the Peer Cloud to see if the information is available before hitting Twitter’s servers. If the user wants to publish a tweet, then all the API has to do is to publish it in the Peer Cloud as well as on Twitter’s servers to be accessible from everyone, even people not using DHTwitter. By sending the information part by part and by sending the latest tweets first, transfer speed would be less of a problem. Also, keeping a user’s friends close to him in the overlay network would make request processing in the Peer Cloud faster.

This would seriously reduce the load on Twitter’s server by distributing the lookup and parsing processing throughout the Peer Cloud. It would reduce the number of server hits by connecting the users together, in the same way they connect in the social layer.

Possible Implementation

For now, this is only an idea. I am not planning on coding an implementation on my own in the near future. However, if someone is interested, please let me know, I would love to participate in the development. The biggest challenge in implementing DHTwitter is the lack of libraries for DHTs. Full system specifications exist; Kademlia, Pastry and Tapestry(Chimera) are good examples. However, there is a lack of solid implementation. Librairies are practically nonexistent and most practical implementations are hard coded into existing applications, such as Vuze or LimeWire. Again, if you thought of a way this could be implemented, please let me know.

Tor
Tor is free software that helps you protect your privacy online. It is an implementation of onion routing that allows you to browse the Internet anonymously. It is a decentralized network, meaning that anybody can create an anonymous server and add it to the network. To make a long story short, onion routing systems work by encrypting packets multiple time using different keys from different server in the network. The packet is then sent to the first server who can decrypt the first layer to find who to send the packet to next. The following servers then do the same thing until the packet reaches its final destination. This way, each server is only aware of the server that comes just before and after it. It is thus impossible for a single server to tell who the packet comes from and what is its final destination. From an ISP point of view, the packet just looks like it is going to a random server and its encrypted content looks like gibberish. Tor is probably the most popular anonymity network and it is the one I was using when I started college. A good front-end for the system is the cross-platform controller Vidalia.

JAP/JonDonym
JAP, the Java Anon Proxy, is similar in design to Tor. It also uses multiple layers of encryption to anonymize the users. The main difference is that JAP servers are not anonymous. When you use the service, you can choose between different Mix Cascades of server. This allows you to choose who you trust and who you don’t. This is a major difference with Tor, with which you don’t know who owns the servers and if they are keeping logs. However, this has a downside. Since everybody knows who owns the servers, this leaves them vulnerable to attacks from hackers or the government. There is also a known backdoor implemented in the software following an intervention by the German Federal Criminal Police Office. It is still a good service though, and one that I used for my last year in college after the IT service found how to block Tor.

I2P
I2P stands for the Invisible Internet Project. It is also a system that keeps its users anonymity using multiple layers of encryption. It is a decentralized network, like Tor. However, it encrypts the data from the beginning to the ends. It even keeps the sender and the destination secret to the middle servers. It can easily be integrated into different software to anonymize their activity over the Internet. Supported protocols include regular TCP/IP communications through I2PTunnel, a simple tunneling application to browse the web, BitTorrent, eDonkey, Gnutella and more. However, it is still considered as beta so there may be some bugs left.

Peer Guardian/MoBlock
Peer Guardian, for Windows and Mac OS X, and MoBlock, for Linux, are open source IP filtering programs. These programs allows you to create a list of IP addresses that you don’t want your computer to connect to. You can easily find extensive lists for anti-piracy groups on the Internet. These lists includes IP addresses from the MPAA, MediaDefender and other groups which use peer-to-peer network to find people to sue for copyright infringement. This way you can make sure you share only with real peers and not people who wants to spy on you.

VPN
If the solutions above are not enough, or if you don’t want to sacrifice speed by going through an anonymous network, then subscribing to a VPN may be a good alternative for you. A VPN, a virtual private network, is similar to a home network except that it sits on the Internet instead of being local. By connecting to the network, all outside requests you make through the network uses the network’s IP address. This means that it is impossible to tell who the original requests come from, only which VPN it comes from. Also, from an ISP point of view, all the requests look like they are directed to the same address and the content of the packets is usually encrypted. An example of a VPN provider is IPREDator, a service offered by the founders of The Pirate Bay. For an extensive list, your can google for VPN providers.

Most file sharing protocols also allow encrypted transfers. Just look through your client’s options. Even though the encryption is pretty weak, it can mislead your ISP filtering system into thinking it is regular traffic. Another useful tool to help you navigate anonymously is FoxyProxy. It is a small Firefox addon that eases the transition from one service to the other. This way you can switch from normal browsing to anonymous browsing in a single click.

So don’t be a dummy and protect your privacy!

But since I am not a professional designer and I can’t write a full post about the characteristic of the layout I chose or the typography concepts in use, I will write about the Wordpress plugins I use in the theme. As you may already know, this blog is driven by Wordpress and plugins play a big part in this CMS. However, I am not a huge plugin user, but there are still 10 that I consider essential to a good blog.

Askimet
The first in my list is the only one that actually comes with a fresh Wordpress install even if it is desactivated by default. Askimet is one of the most powerful and useful plugin. It is a spam filter that becomes especially handy when your blog starts showing up in search results. With approximately 99% of the comments posted on my blog being spam, I can’t live without this one.

All in One SEO Pack
All in One SEO Pack, as its name says, is a plugin that faciliate the search engine optimization of your blog. Using a simple administration interface, you can change the title of your pages with simple rules and it automatically adds various information to your pages headers like keywords or descriptions.

Google Analyticator
I first installed this one in a moment of lazyness. I didn’t want to manually integrate the Google Analytics code into my footer template in case I change theme in the future. However, it is when I visited the settings page that I realised all the power the plugin offers. Not only does it allow you to embed the analytics code automatically in your blog, but it also offers a variety of advanced options to tweak the service to your needs.

Google XML Sitemaps
Google XML Sitemaps is a complement to All in One SEO Pack. Don’t be tricked by its name, it doesn’t only generate and submit your blog’s sitemap to Google; It also does so for Yahoo!, Live Search and Ask.com. This is a really good plugin if you want your blog to come up in search results. It also leaves a lot of place open for the user to customize the creation and the submission processes of the sitemap.

ShareThis
ShareThis is an essential for social bloggers. Like AddThis and other similar services, it makes it easy to embbed links to delicious, digg, Twitter, Facebook and other social services to your posts. The difference between ShareThis and these other services is that it does it better. The way the plugin is designed lets you position it wherever you want and change its appearance to match with your website.

Simple Tags
Simple Tags is a ninja plugin. In no way your user will ever notice you are using it, but that doesn’t mean it is not useful. It allows you to manage your tag really easily in the administration interface. You can mass tag as well as auto tag your posts using different auto tag services. On the front-end side, it offers more functions to display your tags the way you want, whether it is in a list or in a cloud.

SyntaxHighlighter Evolved
This one is a must for coders. SyntaxHighlighter Evolved is the most complete code highlighting plugin I found. It supports a wide array of languages and offers different color schemes to match your design. It also displays your code in a concise and effective way. If you write code often, you need this one.

Twitter for Wordpress
For the Twitter fans out there, this plugins allows you to integrate the twitterfeed of any public profile. It lets you choose what information to grab, what to display and how to display it. Twitter being pretty minimal, the plugin is also pretty minimal, but it does what it should do and it does it well.

Wordpress Automatic Upgrade
Another plugin for the lazy bloggers, Wordpress Automatic Upgrade does what it says it does, it automatically upgrades your wordpress install when a new version comes out. It does so step by step, asking for user input between each step to make sure nothing goes wrong. It also gives you links to backups of your database and system in case something breaks during the update process.

WordPress Related Posts
The final plugin in my top ten list is Wordpress Related Posts. This one is a plus for your reader and yourself at the same time. It displays links to posts similar to the one currently displayed by analysing the tags and categories of each posts. This hopefully keeps visitors for a longer time on your blog, giving them more to read.

So this is it. A new interface and a new awesome list of wordpress plugin. I hope you enjoy your new reading environment and never forget: stay KISS!

First of all, I have to say that I love Twitter. I think it is a really useful tool for the rapid spread of what I would call bleeding edge knowledge. It is easy to find and spread the latest news in a variety of domains, mostly through links to interesting blog posts. My fears are for Twitter Inc, the corporation behind the service.

Lately, Twitter has been opening its API like no proprietary web service before. Back in September, when I started working with their different developer tools, the only option available to me was to use an API key to make requests using a limited sets of functions. Today, with the integration of OAuth to the API and the development of other useful tools, connecting to Twitter has never been easier. It got to the point where the user has dozens of alternative to the official website to connect with its fellow twitterers, some of which are way more advanced than the original web interface or target specific aspects of the Twittersphere. Based on recent statistics, less than 75% of the users post their updates via the web client. The question is: is this a good thing?

From the point of view of the user, it is the best thing that could happen. Opening the API to developers worldwide means faster and better innovation. Twitter’s users are prosumers that don’t passively use the end product, but also develop it based on their needs. The peer pioneering behind Twitter lead to the creation of whole new services, like Twitpic, that integrates seamlessly to the main service via the open API. This means more alternative for the end user and, as a result, a better experience.

The conclusion is the same from the service’s point of view. By opening itself to the world, Twitter gains new features and users everyday, making it more useful and purposeful. Clients like Tweetdeck and Seesmic Desktop make it easy to reorganize and mash the data available on the Twittersphere, keeping only the parts that are relevant. You can easily find what is most discussed by visiting Twitscoop or automatically post with Twitterfeed. It is these services that make the Twitter service more than a 140 characters blog.

The problem is with Twitter Inc. All these API requests have to be managed by some central system, a system that happens to be owned and operated by Twitter Inc. Every request has to go through one of Twitter’s server, creating operation and maintenance costs that they have to pay to stay online. They could have tried to payoff their bills with ads like other Web 2.0 companies, but not only do they don’t display ads, but they soon may not even display any content at all. As I said before, less than 75% of the users use the official web interface. That means that more than 25% of the users don’t even bother going on the Twitter website and this number is growing. By decentralizing a system that, in is nature, is centralized, Twitter Inc is facing the risk of becoming a simple content provider for more advanced clients or at least, to lose potential profits to someone else. Even worse is the fact that its problem, the widespread availability of the content, is also its strength. Even identi.ca, a free and open source micro-blogging platform, seems to have a better business plan with its status.net initiative. So how can Twitter be worth $700 millions? Hype. But hype is not enough in the long run. With a loss of control over the distribution of its content, Twitter will have to find alternative way to start making money or it will become part of the past.

We plan to build Twitter, Inc into a successful, revenue-generating company that attracts world-class talent with an inspiring culture and attitude towards doing business.

About Twitter

I hope for them that this is not just a dream.

Let’s take some big Web 2.0 players such as YouTube or Facebook. If there is one thing that the last past years have proved about these applications, it is that profitability is hard to achieve. Internet Evolution published an article last week about how YouTube will lose more than $2 millions a day next year. Not to talk about Facebook which is still not profitable. So what do they have? What gives Twitter, an application that allows people to post message of less than 140 characters, the right to refuse a $500 millions offer from Facebook? The answer, I think, stands in one word: hype. Web 2.0 applications gets their value from the momentum they have, from the number of users they gather, but it is a good way to value a company?

Lets consider a good old Web 1.0 application, Google. There is no doubt that Google achieved success, but not only in the way Facebook or Twitter achieved success, it is now a really lucrative corporation. So what does Google have that Facebook doesn’t have? I think the main difference lays in the foundation of the Web 2.0 movement: the interaction with the user. Web 2.0 websites are often more service oriented than a classic web application. By that, I mean that the application and the company behind it offers a service to the user in which they usually takes little to no place. For example, YouTube’s user will upload content for other users to watch and he will watch content uploaded by other user. On the other hand, if we take a website like Gamespot or IGN, then the company, which we could call the publisher, takes part into the user experience by creating most the content by itself.

So what does this means? From a user’s point of view, that usually means a less interactive experience. Web 1.0 applications have limitations which don’t exist in Web 2.0. Less interaction usually means less hype, and a shorter attention span. People go to Google to go elsewhere, or go on Gamespot to read. Once this is done, there is no reason to stay on the website. This is a totally unilateral relation. Web 2.0 applications offer a bilateral relation, where the user gets updates more frequently and can interact with other dynamic beings. It is the AJAX of user interaction. A world where the user is at the center and everything branches from him.

However, this is also the weakness of Web 2.0 applications. When in the center, the user often lose interest of the outside world. Targeting someone with advertising on Facebook may not be hard, especially with all the informations available, but the user isn’t there to buy. He is there to interact with other Facebook entities, which, as of now, are largely noncommercial. However, someone that goes on CNET to read on audio devices, or on Google to search for cars, is much more likely to click on an ad. Even if the attention span and the engagement is less, the value of these applications is in the relevance of the content and its commerciality. What could Facebook sells? Friends? Relations? No. With a website that focus on user generated content or social relations, it is really hard to target users effectively, because what they come to your website for is not something you can easily monetize. Especially through ads.

So what is the future of Web 2.0? My guess is Web 2.5: a Web 2.0 with a valid business model that takes advantage of the commitment of the users in their products. When this will be achieved, it may stop the rise and fall of the fragile MySpace, Facebook and Twitter of this world and give the final blow to Web 1.0.