Google have put together a demonstrative video of their iPhone Mobile Voice Search app which was temporarily removed from Apple’s App Store recently, upon the discovery that Google had intentionally broken the API.

Read the rest of this entry »

A new image-based search engine called TinEye has recently been made available to the public, which may prove very useful to web authors, photographers and users of photo sharing sites such as Flickr. The search engine can trace the use of a single image across the internet even if it has been photoshopped.

Read the rest of this entry »

As a part of Google’s pledge to be more open about their business activities, they have revealed information about the inner-workings of their data centres.

Google’s Jeff Dean spoke to a crowd at the Google I/O conference on Wednesday describing how their data-centres are assembled.

Apparently Google uses more-or-less ordinary servers and stacks 40 in each rack. It has not been revealed how many servers it has, but with 150 racks per data centre and 36 data centres around the world we can estimate that they have over 200,000 and growing.

Each Google search query involves 700 to 1000 servers yet manages to return a response within a sub-half second.

Google largely builds its own technology instead of relying on mainstream servers and treats each machine as being expendable. Google prefers to invest in fault-tolerant software instead of top-end hardware.

“Our view is it’s better to have twice as much hardware that’s not as reliable than half as much that’s more reliable,” Dean said. “You have to provide reliability on a software level. If you’re running 10,000 machines, something is going to die every day.”

Bringing a new cluster online shows how fallible hardware can be, Dean said.

In each cluster’s first year, it’s typical that thousands of failures of hard drives will occur; 1,000 individual machine failures will occur and 5 racks will “go wonky” losing half their network packets. The cluster will have to be rewired once which affects at any given moment 5 percent of the machines over 2 days and one power distribution unit will fail bringing down 500 to 1,000 machines for about 6 hours, Dean said.

There’s also a 50 percent chance the cluster will overheat which will bring down most of the servers in less than 5 minutes – taking 1 to 2 days to recover.

There are three core elements to Google’s software architecture, MapReduce, BigTable and Google’s file system called GFS which are all proprietary.

Dean said that GFS runs on almost all machines and stores data on many. Some versions of GFS are “many petabytes (a million gigabytes) in size.”

Structure is provided by BigTable, Google’s database software. High profile commercial database management software from companies such as Oracle or IBM can’t operate on the scale that Google requires. Also, their licenses would cost the company far too much.

Google began creating BigTable in 2004 and it is now used in over 70 projects including Blogger, Google Earth, Google Maps, Google Print, Orkut as well as their massive search index.

MapReduce was created in 2003 to make good use of Google’s data. It can find how many times a single word shows up in their index and create a list of all websites that link to any given website.

Like GFS, MapReduce is designed to sidestep server problems. One system, during a presentation in 2004, withstood a failure of 1,600 servers out of a cluster of 1,800, Dean said.

As always, there are many projects in the works at Google, a company that never sits still. Hopefully they will keep their new open policy and continue to share with us information of their fascinating work and achievements.

Sources:
CNet News

Microsoft’s attempt to scan entire libraries in an effort to make the information searchable, will now be abandoned.

It had amassed digital copies of 750,000 books and 80 million journal articles but was under immense competition from market leader Google, who runs a similar service but does not restrict itself to scanning works in the public domain.

Microsoft began their book-scanning business in 2005 and unveiled their MSN book search site in 2006. They stuck to copying only books and documents that were firmly in the public domain, working with publishers to scan their books.

Two separate sites for searching through the content will be taken down and the massive collection of documents will be given to publishers. Live Search will direct users wanting to search for books to non-Microsoft sites, the company said.

The world’s biggest software maker is being pressured to show how it will turn around its unprofitable online business after its collapsed bid for search competitor Yahoo earlier this month.

Satya Nadella, Senior Vice President wrote in a blog post on Friday that digitizing books and archiving academic journals no longer fits with the company’s plan for its search operation.
The move will allow Microsoft to focus on other types of Internet searches, such as travel listings, she wrote.

“We believe the next generation of search is about the development of an underlying, sustainable business model for the search engine, consumer and content partner,” wrote Nadella.

Last week, Microsoft announced its new Live Search Cash Back system that they hope will gain them a bigger share of the search market by offering a Cash Back incentive to users who make purchases on advertisers’ sites. Microsoft is a distant third in the search market behind Yahoo and market leader Google.

Google’s book project began in 2004, working with the New York Public Library, Harvard University and other organisations for the mass scanning of books. Though they haven’t detailed a way to make book searches profitable, they could run ads alongside search results much as they do with traditional searches. There is also the option of a profit share with publishers when books are sold.

Photo credit: Stephen Brashear – AP