tag:blogger.com,1999:blog-6314876008291942531.post2942800840541659142..comments2024-01-14T00:36:43.430-08:00Comments on Antonio Gulli's coding playground: Magy!: how many servers for a video search engine?Unknownnoreply@blogger.comBlogger3125tag:blogger.com,1999:blog-6314876008291942531.post-78383584553255709972009-01-13T03:32:00.000-08:002009-01-13T03:32:00.000-08:001) correct. but you may want to estimate an averag...1) correct. but you may want to estimate an average case as well. Not all the videos will reach 30Mbytes.<BR/><BR/>2) Suppose that you do want to use commodity low cost servers and implement a software redundancy.<BR/><BR/>3) Not all the videos will be accessed with the same frequency. A caching layer will improve a lot the qps of the systemcodingplaygroundhttps://www.blogger.com/profile/08478993186814330588noreply@blogger.comtag:blogger.com,1999:blog-6314876008291942531.post-19479516298418652562009-01-12T21:56:00.000-08:002009-01-12T21:56:00.000-08:00Err, did I won something? :-)Err, did I won something? :-)Taz70https://www.blogger.com/profile/01108038671156290798noreply@blogger.comtag:blogger.com,1999:blog-6314876008291942531.post-10634237609375381472009-01-12T21:55:00.000-08:002009-01-12T21:55:00.000-08:001) First questionLet's say that you have 31 Millio...1) First question<BR/>Let's say that you have 31 Million of seconds a year, and you define 30 megabytes to be the maximum uploaded video's size.<BR/>Doing the math this gives a total of 473,040 Tera a year, or 473 Petabyte of some kind of redundant storage.<BR/><BR/>2) Second question.<BR/>What kind of infrastructure can accomodate 473 Peta of storage? <BR/>I'm my opinion that's kind of infrastructure could not still be built. But, well it depends on you budget actually. :-)<BR/>You would need a sum of 1000 EMC DMX-4 expanded to maximum capacity, using and enterprise storage solution, or an amount of about 100k standard Intel Server equipped with 5 1 Tera disk in raid configuration.<BR/>Did you were thinking about something particular ?<BR/><BR/>3) Third QUestion.<BR/>Caching ? Who need caching when you have a such massive and disperse base of data. Unless you meant it for the CDN...Taz70https://www.blogger.com/profile/01108038671156290798noreply@blogger.com