Google-MegaStore的解读

八月 14th, 2013 by klose | 2 Comments | Filed in 互联网应用, 海量数据存储与处理

MegaStore是Google在BigTable之上实现了一个跨机房高可用的数据库。 它提供了类似DB的数据分布、索引的功能,实现了在EntityGroup内部以及EntityGroup之间的事务性,并且通过Paxos协议实现在DC之间多备份的一致性。 MegaStore的目标:在跨机房PB级的数据规模上,支持交互式在线服务。我们知道在Google内部的访问情况是,每天几百亿次的访问请求的应用,读写比例大概在7:1。在这样的规模上,需要达到的特性: 1)高扩展性,无论在机器扩展性还是数据规模上 2)快速的开发迭代速度, 3)低延迟 4)数据一致性 5)高可用 MegaStore的实现原理: 1)从数据库的扩展性的角度上,把大数据切分成更小粒度的数据集(EG),每个数据库保持一个独立的log,保存在BigTable当中。//这也就意味着在每个DC上有一个独立的BigTable数据库。 2)从可用性角度上讲,在跨机

Tags: , , ,

Google Spanner架构

一月 17th, 2013 by klose | No Comments | Filed in 互联网应用, 海量数据存储与处理

Spanner is a scalable, globally-distributed database designed, built, and deployed at Google. At the highest level of abstraction, it is a database that shards data across many sets of Paxos state machines in datacenters spread all over the world. Replication is used for global availability and geographic locality; clients automatically failover between replicas. Spanner automatically reshards data across machines as the amount of data or the number of servers changes, and it automatically migrates data across machines (even across datacenters) to balance load and in response to failures. Spanner is designed to scale up to millions of machines across hundreds of datacenters and trillions

Tags: ,