g*****g
发帖数: 34805 | 1 一开始扯什么4网卡全双工80GB网速，网络极限是唯一瓶颈。同时又扯优化后的协议，
每个request几百bytes。我粗粗一算，每秒能支持1亿次写，预算只要一万，我一看，
这性能可以秒了银河，要求给个benchmark看看。魏老师一看扯过了，立马改成了内存
数据库，总之先放内存里，慢慢往硬盘上写，本来觉得可有可无的hot standby，现在
自己强烈要求上了。
所以到头来，魏老师设计了个内存数据库，局域网上一个standby，这栋楼一断电，全
玩完，不只是交易中断，完成的交易也要丢。打死我都不信任何一个stock exchange敢
这么做服务来提高throughput的，魏老师就一大忽悠，居然还有人鼓吹他思路好。 |
g*****g
发帖数: 34805 | 2 再说我为啥把魏老师逼到这角落里。这个系统，throughput最高的地方，在于订单。
魏老师的方案就是单机+standby+拼命优化，我的方案就是简单的Cassandra cluster
based MQ。
对于这么一个系统，受到冲击的时候，每秒10万次写是必须达到的，每秒100万次写都
有可能在短时间内达到。这用一个像Cassandra的distributed DB是可以做到的，而用
单机，IBM最大的机器拿过来有可能能行，这也就是我一开始就说你买一太服务器的钱
，够我这系统EC2上跑10年。但魏老师非装逼要用1万刀的commodity hardware。被打脸
是自找的。 |
x****d
发帖数: 1766 | 3 大家不都是看pornhub看烦了来打打嘴炮么，过瘾就行，看人家撸管还要非要指导人家
手法，那就过了，大家哈哈乐乐就好，买买提又不出工资，又不是学术论文发表的地方
，有人来歪歪大家聊聊就挺好。。。 |
b***e
发帖数: 1419 | 4 Assume there is a specific ticket that is very hot and can expect a ton of
concurrent purchase attempt. Also assume you cannot use the offline
processing ultimate weapon, which forces you to do on the fly quantity
update, how would you tackle the massive concurrent write?
【在 g*****g 的大作中提到】
: 一开始扯什么4网卡全双工80GB网速，网络极限是唯一瓶颈。同时又扯优化后的协议，
: 每个request几百bytes。我粗粗一算，每秒能支持1亿次写，预算只要一万，我一看，
: 这性能可以秒了银河，要求给个benchmark看看。魏老师一看扯过了，立马改成了内存
: 数据库，总之先放内存里，慢慢往硬盘上写，本来觉得可有可无的hot standby，现在
: 自己强烈要求上了。
: 所以到头来，魏老师设计了个内存数据库，局域网上一个standby，这栋楼一断电，全
: 玩完，不只是交易中断，完成的交易也要丢。打死我都不信任何一个stock exchange敢
: 这么做服务来提高throughput的，魏老师就一大忽悠，居然还有人鼓吹他思路好。
|
g*****g
发帖数: 34805 | 5 If that's the case, nothing stops you from splitting tickets in DB, instead
of one DB with 10K tickets, you can have 10 DB with 1K tickets each. And a
cluster of load balancers can poll the DB, get the number of remaining
tickets, and distribute the requests accordingly to avoid hotspot.
【在 b***e 的大作中提到】
: Assume there is a specific ticket that is very hot and can expect a ton of
: concurrent purchase attempt. Also assume you cannot use the offline
: processing ultimate weapon, which forces you to do on the fly quantity
: update, how would you tackle the massive concurrent write?
|
b***e
发帖数: 1419 | 6 Good. But then
1. I feel I don't a centralized source of truth.
2. I might want the flexibility to adjust the number in each bucket.
how do you improve?
instead
【在 g*****g 的大作中提到】
: If that's the case, nothing stops you from splitting tickets in DB, instead
: of one DB with 10K tickets, you can have 10 DB with 1K tickets each. And a
: cluster of load balancers can poll the DB, get the number of remaining
: tickets, and distribute the requests accordingly to avoid hotspot.
|
g*****g
发帖数: 34805 | 7 For 10K tickets, all tickets are gone before you need any adjustment.
Generally speaking however, if the volume is very high, say 10M.
You can have a service to re-adjust the buckets, you would only want to
adjust 2 buckets at a time on the max and min buckets when certain threshold
is reached. A distributed
transaction is needed and both buckets are locked during adjustment.
Alternatively, there're some other ways of doing distributed counters. e.g.
Cassandra.
http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf
You do have the number of total tickets, which is never really real-time
anyway.
【在 b***e 的大作中提到】
: Good. But then
: 1. I feel I don't a centralized source of truth.
: 2. I might want the flexibility to adjust the number in each bucket.
: how do you improve?
:
: instead
|
d********u
发帖数: 5383 | 8 你技术上没有问题，可是解决问题的能力实在需要提高。对于问题不是去分析，而是上
来就拿别人做好的框架往上套。就好像在学校读书的时候，有些同学只会死记硬背，其
实根本没理解。
另外，WEB只是IT技术的一个分支，不幸的是，还是技术低端的一个分支。有些人竟然
以为WEB==IT,太可悲了。
我想这就是为什么A3是老板，你是骡子的原因。
【在 g*****g 的大作中提到】
: 一开始扯什么4网卡全双工80GB网速，网络极限是唯一瓶颈。同时又扯优化后的协议，
: 每个request几百bytes。我粗粗一算，每秒能支持1亿次写，预算只要一万，我一看，
: 这性能可以秒了银河，要求给个benchmark看看。魏老师一看扯过了，立马改成了内存
: 数据库，总之先放内存里，慢慢往硬盘上写，本来觉得可有可无的hot standby，现在
: 自己强烈要求上了。
: 所以到头来，魏老师设计了个内存数据库，局域网上一个standby，这栋楼一断电，全
: 玩完，不只是交易中断，完成的交易也要丢。打死我都不信任何一个stock exchange敢
: 这么做服务来提高throughput的，魏老师就一大忽悠，居然还有人鼓吹他思路好。
|
g*****g
发帖数: 34805 | 9 买辆好车，你是弄个好的加定制，还是自己从轮子开始DIY?
我老对问题没分析？我老把数据是怎么样的，可以怎么分的办法至少提了三种，无脑就
说数据就是不能分的
才是真的悲哀。
【在 d********u 的大作中提到】
: 你技术上没有问题，可是解决问题的能力实在需要提高。对于问题不是去分析，而是上
: 来就拿别人做好的框架往上套。就好像在学校读书的时候，有些同学只会死记硬背，其
: 实根本没理解。
: 另外，WEB只是IT技术的一个分支，不幸的是，还是技术低端的一个分支。有些人竟然
: 以为WEB==IT,太可悲了。
: 我想这就是为什么A3是老板，你是骡子的原因。
|
b***e
发帖数: 1419 | 10 Good. This is a demonstration of CAP thesis, where we do have availability (
on-the-fly transaction) and partition, but compromise consistency (real time
accuracy of quantity).
Here's my point of view: architectures, like design patterns or algorithms,
are not bound to Java or any other languages. It can certainly be discussed
independently. You don't have to try your best to sell Java all the time.
I believe you can have more people agreeing with you that way.
threshold
.
【在 g*****g 的大作中提到】
: For 10K tickets, all tickets are gone before you need any adjustment.
: Generally speaking however, if the volume is very high, say 10M.
: You can have a service to re-adjust the buckets, you would only want to
: adjust 2 buckets at a time on the max and min buckets when certain threshold
: is reached. A distributed
: transaction is needed and both buckets are locked during adjustment.
: Alternatively, there're some other ways of doing distributed counters. e.g.
: Cassandra.
: http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf
: You do have the number of total tickets, which is never really real-time
|
|
|
g*****g
发帖数: 34805 | 11 I didn't mention Java in the entire ticketing system design. I just don't
believe such a system can be done on single DB, especially the part taking
orders.
(
time
,
discussed
.
【在 b***e 的大作中提到】
: Good. This is a demonstration of CAP thesis, where we do have availability (
: on-the-fly transaction) and partition, but compromise consistency (real time
: accuracy of quantity).
: Here's my point of view: architectures, like design patterns or algorithms,
: are not bound to Java or any other languages. It can certainly be discussed
: independently. You don't have to try your best to sell Java all the time.
: I believe you can have more people agreeing with you that way.
:
: threshold
: .
|
b***e
发帖数: 1419 | 12 On the ticketing system design I am certainly biased to agree with you,
given my background.
The comments on Java is more on a general sense. Look, in many cases you do
have good points, but there're two problems:
1. Attitude.
2. 夹带私货
If I were to choose between:
1. to show how 牛逼 I am
2. to have people agree with me and follow me
I would opt for the latter. The former might make you a good architect (
which I believe you are), while the latter makes you a good leader.
【在 g*****g 的大作中提到】
: I didn't mention Java in the entire ticketing system design. I just don't
: believe such a system can be done on single DB, especially the part taking
: orders.
:
: (
: time
: ,
: discussed
: .
|
g*****g
发帖数: 34805 | 13 您老要跟我谈态度先照照镜子。我老说话难听好歹绝大多数时候还是谈技术，
你跟赵策光互相问候家人就问候了一晚上，纯的。
do
【在 b***e 的大作中提到】
: On the ticketing system design I am certainly biased to agree with you,
: given my background.
: The comments on Java is more on a general sense. Look, in many cases you do
: have good points, but there're two problems:
: 1. Attitude.
: 2. 夹带私货
: If I were to choose between:
: 1. to show how 牛逼 I am
: 2. to have people agree with me and follow me
: I would opt for the latter. The former might make you a good architect (
|
b***e
发帖数: 1419 | 14 找厕那个是找抽型的。我一般也不发水贴。不过说到态度我也不算好。没必要。
【在 g*****g 的大作中提到】
: 您老要跟我谈态度先照照镜子。我老说话难听好歹绝大多数时候还是谈技术，
: 你跟赵策光互相问候家人就问候了一晚上，纯的。
:
: do
|
z****e
发帖数: 54598 | 15 哈哈，就是看你那边时间太迟了，故意拖着你，让你熬夜
然后差不多到点了就洗澡睡觉了，你熬夜熬了一个晚上是不是很high啊？
【在 b***e 的大作中提到】
: 找厕那个是找抽型的。我一般也不发水贴。不过说到态度我也不算好。没必要。
|
l*****9
发帖数: 9501 | 16 goodbug就会吹牛
发信人: goodbug (好虫), 信区: Programming
标 题: Re: Obamacare website
发信站: BBS 未名空间站 (Fri Nov 22 19:34:48 2013, 美东)
lol, scalability is never an issue on front tier. |
p*****2
发帖数: 21240 | 17
貌似他比你起的早很多。
【在 z****e 的大作中提到】
: 哈哈，就是看你那边时间太迟了，故意拖着你，让你熬夜
: 然后差不多到点了就洗澡睡觉了，你熬夜熬了一个晚上是不是很high啊？
|
b***e
发帖数: 1419 | 18 你个傻逼我一天不骂你你就逼痒是吧。
【在 z****e 的大作中提到】
: 哈哈，就是看你那边时间太迟了，故意拖着你，让你熬夜
: 然后差不多到点了就洗澡睡觉了，你熬夜熬了一个晚上是不是很high啊？
|
z****e
发帖数: 54598 | 19 放心，等你今晚两点多的时候，我肯定奉陪
操你妈
【在 b***e 的大作中提到】
: 你个傻逼我一天不骂你你就逼痒是吧。
|
d********u
发帖数: 5383 | 20 你第一句话就暴露了：买辆好车
你要先搞清楚，是不是要买车。人家要买红薯，你非要用买车那个流程来生搬硬套。我
知道你对买车是非常专业，但是，人家真的是要买红薯。。。
【在 g*****g 的大作中提到】
: 买辆好车，你是弄个好的加定制，还是自己从轮子开始DIY?
: 我老对问题没分析？我老把数据是怎么样的，可以怎么分的办法至少提了三种，无脑就
: 说数据就是不能分的
: 才是真的悲哀。
|
m*******l
发帖数: 12782 | 21 说一句,每年的感恩节,红薯都打折...
今天偶去买了50斤
【在 d********u 的大作中提到】
: 你第一句话就暴露了：买辆好车
: 你要先搞清楚，是不是要买车。人家要买红薯，你非要用买车那个流程来生搬硬套。我
: 知道你对买车是非常专业，但是，人家真的是要买红薯。。。
|
