Sunday, August 15, 2010

Haystack the Facebook Fileserver

Haystack is the Facebook fileserver for photos. Instead of using costly NFS operations (as they did in the past) they adopted a simple log based file where the typical operation is an append to an already open file. Obviously they have a separate index structure, and many cache server in front of the system.

This is a typical solution when you have a lot of new objects inserted in the system (and a limited amount of delete operation). If you have many delete, you probably should consider something like the COSS (Cyclic Object Storage System) developed for Squid.


  1. This is a nice example of building something with the minimum set of features necessary to get the job done. Sometimes less is more.

  2. This is a typical solution for a search engine when you need to serve documents. DocStore is typically a log based append store.

    BTW, this is presentation for Haystack check it out since it is quite interesting

  3. That doesn't change my point, Antonio. Facebook has a pattern of building the minimum set of features necessary to solve the problem at hand. Microsoft could learn from that.