I thought it would be worth pulling out the "assumptions that are now invalid" from Steve Loughran's and Julio Guijarro's presentation "Farms, Fabrics and Clouds".
- Systems have a long lifespan
- It is slow/expensive to create a new system
- It is expensive to duplicate one
- Systems can/should be managed by hand
- Clocks proceed at the same rate
- Physical RAM doesn't get swapped out
- Running machines can't be moved/cloned
- System failure is an unusual event
- 100% availability can be achieved
- Data is always near the server
- You need physical access to the servers
- Databases are the best storage form
- You need millions of $/£/€ to play
- Terabyte datasets are hard to work with
- Code runs on a single machine
- Sequential code is better than parallel code
- RAID hardware is the best way to store data
- Databases are better than filesystems
- A single farm needs to scale to infinity
- You need to provide 100% availability to 100% of users
- You have to roll out simultaneous updates to the application, changes to the DB schema, globally
This is a HP labs paper, though what's most interesting to are how these clouds are been driven by consumer facing companies (such as Amazon and Google), that claim the enterprise state of art won't cut it at scale. This seems to me like the desktop computer revolution of the eighties or the email/web of the ninties - consumer tech that infects the enterprise. Or maybe that's selection bias from reading blogs and silly-valley articles; perhaps there's a ton of interesting stuff happening in the IT/Datacenter sectors that doesn't get aired much. Another interesting thing is that my thinking about capacity planning and rollout has to change. As Steve says - "you no longer have to estimate load in advance [...] and you have to embrace this dynamic world from the outset." That seems true to me; legacy apps and containers making strong design assumptions around N=1 datasets have been the main reason I see "just scaling out" gets taken off the table as an option - too much valley to cross to the next local maxima; the path of least resistance is a bigger database. Sometimes I wonder how a deployed n-tiered scale up monolith can be gradually refactored to a scale out model or run on scale out infrastructure. It's well documented that companies like Amazon and Ebay have done just that, only how they did it tends to get left out of the slide decks. I suspect it involves thinking quite differently about what 'good' application code is; in that sense I have a lingering doubt about coding direct to an ORM, but less so about service layers.
I would have added one other invalidation:
- virtual hosts are only for testing and development
17 Comments
I really think we'd be a lot further along in the world if every developer were forced to be actually deploy and monitor their own code. Better yet, be a sysadmin for a year or two and do it for other people's code.
Anyway.
@Mark - well said!
@dehora - you know where I'm coming from on that :)
@Mark: yes!
@Murf: you already *know* I am so maintenance!
Hi Bill
"It's well documented that companies like Amazon and eBay have done just that, only how they did it tends to get left out of the slide decks..."
I happen to be in the last Qcon event in which eBay, Amazon, Yahoo presented how they scale their applications - you can find a summary of what i saw as a repetitive patterns for scaling here: http://natishalom.typepad.com/nati_sh... i happen to give a specific presentation during that same event on "how to transition existing tier based application into scale out model" - the presentation is available online here http://qcon.infoq.com/sanfrancisco/pr...
Below is short snippet from my summary on how Amazon,Yahoo,eBay scale their applications:
- Asynchronous event-driven design: Avoid as much as possible any synchronous interaction with the data or business logic tier. Instead, use
an event-driven approach and workflow
- Partitioning/Shards: You need to design your data model so
that it will fit the partitioning model
- Parallel execution: Parallel execution should be used to
get the most out of the available resources...
- Replication (read-mostly): In read-mostly scenarios
(LinkedIN seems to fall into this category well), database replication can help
load-balance the read load by splitting the read requests among the replicated
database nodes
- Avoid the use of distributed transactions....
- Move the database to the background: There was violent
agreement that the database bottleneck can only be solved if database
interactions happen in the background.
Quoting Werner Vogel (Amazon) "To scale: No direct access to the database anymore. Instead data access is encapsulated in services (code and data together), with a stable,
public interface."
HTH
Nati S.
Nati, thanks for the links! I guess I should have been more clear - the what and the techniques I'm aware of, I'm much much fuzzier on the organisational/project processes of doing so.
20100226SLLYP A great mbt shoes poem is a fountain forever overflowing with the waters of wisdom and delight.Poet I mbt shoes sale who puts up a ladder to a star and climbs it while playing a violin.Humor has been well defined as thinking in mbt walking shoes while feeling in earnest.Words have a magical power. They can bring mbt shoes review either the greatest happiness or deepest despair; can transfer knowledge from teacher to students words enable the mbt lami sale orator to sway his audience and dictate its decisions. Words are capable of cheap mbt shoes arousing the strongest emotions and prompting all sell mbt shoes actions. Do not ridicule the use of words in psychotherapy. http://www.mbtshoessaletoday.com/revi...
Welcome to coach outlet, we offer you the best discount coach purses and coach handbags here.
http://bagworlds.com/
Thank you! I would now go on here every day!
http://www.infls.com
We are supply all kinds brand goods:
Shoes: Jordan, Nike,http://www.tougg.com Prada, Puma, LV, Timberland, Pual Smith, D&G, Supra, Adidas, Yeezy, ATO, CR etc.
Clothing: Coat, Jacket, T-shirt, Suit, Sweater, Jeans, Bikini, Jersey etc.
Handbags: Coach, Berberry, Balenciaga, http://www.bayugg.comJuicy, LV, ED-Hardy, MIUMIU, Chanel, Chole etc.
Accessories: Belts, Sunglasses, Hats, Necktie,http://www.tougg.com Scarf, Jewelry etc.
As the commodities we supply are of excellent quality http://www.aheadtrade.com and low price,
we have won a very good reputation from our clients all over the world.
If you are interested in any of our products, please don't hesitate to contact with me by email.
I trust that through our cooperation we shall be able to conclude some transactions with you in the near future.
Website: http://www.mbtshoe.co.uk
http://www.aheadtrade.com
http://www.bayugg.com
http://www.tougg.com
Best regards!
coach outlet. coach handbags and coach purses are sold at best price.
http://bagworlds.com/
100308SLHYZ
A most exciting new style ugg classic cardy for the Ugg line!Side zip on inner calf offers chanel handbags fresh style and easy on and off. Cute ugg short boots flaunts a soft suede upper and smooth seams. coach handbags Genuine sheepskin sockliner naturally wicks moisture away for dry, happy feet. Free shipping and fast delivery to your door. As a online shopping platform,ugg bailey button we have professional sales team, service team and securtiy team. We can provide fashion gucci handbags design and high quality online for our customers. http://www.officalugg.com/products_al...
thanks for you to approach this <a href="http://www.shoesblog.org.uk"title="shoe">shoe</a>.
100309SLLQY
Woman will begin with bailey button uggs certainties, she shall end in ugg classic cardy sale doubts,but if he will be content to begin with louis vuitton handbags sale doubts, he shall end in certainties. Determined to act prada handbags 2009 Decisivly and to bear the consequences.In this cheap coach handbags world, hesitation can not access to success. Laziness as a lock,which bolts you out of the storehouse of mui mui handbags information and makes you an intellectual starveling.Early to guess handbags sale bed and early to rise jimmy choo handbags makes a man healthy,wealthy,and wise. http://www.hideboots.com/products_new...
<p><a href="http://www.ghdssite.com/"><strong>cheap GHD hair straighteners</strong></a></p>
<p><a href="http://www.ghdssite.com/"><strong>GHD straighteners wholesale</strong></a></p>
<p><a href="http://www.ghdssite.com/"><strong>GHD styler</strong></a></p>
<p><a href="http://www.ghdssite.com/"><strong>GHD IV styler</strong></a></p>
<p><a href="http://www.ghdssite.com/ghd-babyliss-hair-straighteners-p-17.html"><strong>GHD Babyliss Hair Straighteners </strong></a></p>
http://www.onlypuma.com puma shoes
http://www.onlypuma.com/mens-puma-sf-... Puma - SF Drift Cat
http://www.onlypuma.com/mens-puma-bay... Puma Baylee Future Cat II
http://www.onlypuma.com/mens-puma-dos... Puma Doshu Combat Shoes
http://www.onlypuma.com/mens-puma-flu... Men's Puma Fluxion II
http://www.onlypuma.com/mens-puma-fut... Puma Future Cat GT Ferrari
http://www.onlypuma.com/mens-puma-fut... Men's Puma Future Cat Lo Engine
http://www.onlypuma.com/mens-puma-fut... Puma Future Cat Low
http://www.onlypuma.com/mens-puma-laz... Puma Lazy Insect
http://www.onlypuma.com/mens-puma-sf-... Puma SF Trionfo Low
http://www.onlypuma.com/mens-puma-spe... Puma Speed Cat
http://www.onlypuma.com/mens-puma-spe... Puma Speed Cat Big
http://www.onlypuma.com/mens-puma-tri... Puma Trionfo Lo L II
http://www.onlypuma.com/mens-puma-tri... Puma Trionfo Low BAYLEE
http://www.onlypuma.com/mens-speed-ca... Speed Cat Big
http://www.onlypuma.com/puma-ducati-t... Puma Ducati Testastretta
http://www.onlypuma.com/puma-kimi-rai... Puma Kimi Raikkonen shoes
http://www.onlypuma.com/womens-puma-b... Puma Basket Brights
http://www.onlypuma.com/womens-puma-b... Puma Baylee Future Cat
http://www.onlypuma.com/womens-puma-d... Puma Drift Cat
http://www.onlypuma.com/womens-puma-e... Puma Engine Cat Low
http://www.onlypuma.com/womens-puma-e... PUMA Espera Patent FS
http://www.onlypuma.com/womens-puma-f... Puma Ferro Tessuto Shoes
http://www.onlypuma.com/womens-puma-f... Puma Future Cat Lo
http://www.onlypuma.com/womens-puma-f... Puma Future Cat Low
http://www.onlypuma.com/womens-puma-r... Puma Repli Cat
http://www.onlypuma.com/womens-puma-s... Puma SF Drift Cat
http://www.onlypuma.com/womens-puma-s... Puma Speed Cat Big
http://www.onlypuma.com/womens-puma-s... Puma Speed Cat Suede
<A href="http://www.patekphilippewatches.us/Rolex/Day%20Date%20II/">Rolex Day-Date II watch for sale</A><BR><A href="http://www.patekphilippewatches.us/">replica watches</A><BR><A href="http://www.patekphilippewatches.us/Montblanc/">replica Montblanc</A><BR><A href="http://www.patekphilippewatches.us/Maurice-Lacroix/">Maurice Lacroix watch for sale</A><BR><A href="http://www.patekphilippewatches.us/Breguet/">Breguet replica</A><BR><A href="http://www.patekphilippewatches.us/Movado/">replica movado</A><BR>
100311SLTH
There are things to be said about such UGG Bailey Button matters, and they are important things, but they belong to a different UGG Classic Cardy order from the things that I wish to say. My UGG Classic Short purpose is to suggest a cure for the ordinary Chloe Leather Handbags day-to-day unhappiness from Ugg Classic Tall which most people in civilised countries suffer, and which Gucci Handbags Sale is all the more unbearable bscause,having no obviious external Ugg Classic Agrly Knit cause, it appears inescapable. http://www.ladyugs.com/specials.html
http://www.nowghd.com/limited-edition... A woman seldom asks advice before she has bought her wedding clothes. I married beneath me. All women do. Any intelligent woman who reads the marriage contract, and then goes into it, deserves all the consequences. http://www.nowghd.com/ghd-babyliss-ha...