I thus decided to implement these simple, but oh so useful data structures. Sure, you could reply that ruby arrays already implement all the functionalities provided by theses data structures. True, but when you are dealing with linked lists of millions of elements, having to relocate the whole array in memory to add an element can prove to be a lenghty task. Also, finding a memory chunk big enough to hold the whole array may prove challenging. Ruby arrays grow pretty fast (~1.5x) as can be seen from the following code snippet from array.c (ruby 1.9.1p243) and can rapidly become hard to store in memory.

if (idx >= ARY_CAPA(ary)) {
    long new_capa = ARY_CAPA(ary) / 2;

    if (new_capa < ARY_DEFAULT_SIZE) {
        new_capa = ARY_DEFAULT_SIZE;
    }
    if (new_capa >= ARY_MAX_SIZE - idx) {
        new_capa = (ARY_MAX_SIZE - idx) / 2;
    }
    new_capa += idx;
    ary_resize_capa(ary, new_capa);
}

And finally, theses data structures implement functionalities that are badly supported by arrays such as deleting or adding an element in the middle of a list.

The stack and queue implementation are separated from the linked list implementation to keep them as simple as possible. The linked list implementation is more extensive and is also used to implement the sorted linked list data structure. The next data structure to be added to Batfish will probably be graphs and it should include a number of basic graph algorithms.

Trie

A trie is a data structure that is used to store an associative array where the array’s keys are strings. It has the same structure as any other tree, except that keys are not stored in nodes. Instead, each edge has a character associated with it and you browse the trie by going down the edges, one character at a time, until you reach the end of the key. The node you reach this way contains the value associated with the key.

Tries have several advantages over binary search trees. First, the complexity of trie lookup is O(L) where L is the length of the key while it is of O(n) where n is the number of elements in the tree for a BST. It also takes less space since different keys overlap. It also have advantages over hash tables. First, the keys, in a trie, are ordered, which makes it a useful data structure to use to store a dictionary. It can also lead to faster lookup depending on the hash function and considering that collisions are possible with string hashes.

More informations on tries can be found here.

BK-Tree

A BK-tree is a useful data structure for nearest neighbor lookup in discrete metric spaces. A metric space is any space that obeys the following rules, where d(a,b) is the distance between a and b.

The later is also known as the triangle inequality. It basically states that there is no shorter way to go from a point to another than the direct way. Examples of discrete metric spaces, that is, where the distances are integers, are the real numbers or the levenshtein distance between strings.

The BK-Tree is constructed by measuring the distance between the value to insert and every node, going down the edges corresponding to the distance at each node. Once an unregistered distance at a node is calculated for that node, the value is attached to it. The lookup process works by going down each edges in the distance to the node ± the lookup treshold range until a node with a distance equal to the treshold value or less is found. It is thus possible to find all the nodes within a certain distance of a value without going through each nodes. However, the larger the threshold, the more nodes you have to visit.

For more information on BK-trees, you can read the following article.

If you are interested in implementing any other functionality or website, don’t hesitate to fork the project on github. Here is the code of the Download superclass.

# The Download superclass
class Download
 attr_accessor :url, :name

 # Initializes a download
 def initialize(url, opts)
 @url = url
 @opts = opts
 @file = nil
 end

 # Tests the url
 def test
 begin
 Net::HTTP.get_response(URI.parse(url)).value
 rescue
 return false
 end
 return true
 end

 # Executes the download
 def execute
 self.set_file
 self.download
 end

 # Sets the file for the download
 def set_file
 @file = @url
 end

 # Downloads the file
 def download
 cmd = "wget -c --user-agent="#{@opts[:header]['User-Agent']}""
 cmd += " -q" if !$verbose
 cmd += " --directory-prefix="#{@opts[:dir]}"" if @opts[:dir]

 IO.popen(cmd + " #{@file}"){|f|}
 end
end

I also received my marks for my first term and I have to say that I am pretty happy with what I got. This term is more focused on computer science related subjects with courses such as Functional Programming and Data Abstraction (using Scheme and C++) and Digital Circuits and Systems. I also have two economy classes which is something I enjoy a lot. It is also the beginning of the hiring process for this summer co-op term.

And, as of right now, I am reading books on ruby and ruby on rails to improve my skills. I did a lot of small scripts and projects using this language/framework, but the MVC concept is still new to me. I am now looking at creating a complete, usable web application so I need to expand my knowledge a bit more. Stay tuned for more.

Also, last thursday was the Engineering Student Awards Dinner. As an entrance scolarship recipient I was invited with some 400 other engineering students. At my table was sitting Gaurav Jain, who graduated from software engineering last year and is now the VP of Polar Mobile, a young and quite successful company developing a smartphone publishing tool. I had the occasion to talk a bit with him about how he started the company and how he is managing it. He is a pretty brilliant guy and his company looks really promising.

I’m not working on a particular project. I discovered Project Euler and spent some time doing the first 10 challenges in C. I’m currently learning Ruby and the Ruby on Rails framework via tutorials. I have an idea in mind for a future project that would fit perfectly with Rails but I’m not proficient enough right now to do anything effective. But I’m a quick learner so stay tuned!