Cloud - Web and Service Cloud

Microsoft launched the Azure cloud based platform at the PDC today.  Microsoft has launched many file storage services that were their cloud offerings to date in Mesh, Foldershare, Groove and more.  Azure is what appears to be a real cloud platform to compete with Amazon and Google rather than just storage hosting.

Build new applications in the cloud - or use interoperable services that run on Microsoft infrastructure to extend and enhance your existing applications. You choose what’s right for you.

It appears so far that it is pretty Microsoft centric for tool support.  Of course the software and servers will be Windows.  This week and last, Microsoft platforms have made their way into the cloud platforms at Amazon and now Microsoft.  Google also recently announced the support of Java.  Another set of aquisitions at Rackspace in the buying of Slicehost and JungleDisk also seem to show the space heating up and the companies all believing in the cloud platform emergence and evolution that seems to be happening.

Amazon EC2 is officially out of beta, it is about time some of these services actually launched.  It is hard to convince people to use the cloud layer without being out of beta (AppEngine when’s it gonna happen huh?).

Amazon also launches with windows support, SQL Server support and much more.  This is great news in times where budgets are tight and people want to start scalable businesses but want to only pay for what is used.  The cloud layer will be a very attractive option to many.

Learn more about the Elastic Computing Cloud (EC2) at Amazon. There are already lots of great simple toos like ElasticFox (Firefox EC2 Extension) to help manage your AMIs from a browser.  You can start and stop armies of configured servers from a little extension in your browser.

Developers are getting many tools to build great things.  We hope more products are out of beta soon like AppEngine.

Google AppEngine is alot of fun, not only is it a good excuse to use Python but it is a touch of the future and lots of possibilities for programmers and engineers from all sized businesses to use. When they opened the gates I was one of the lucky 10,000 to get in. So what did I do, I setup three apps before I even knew that was the limit.  Then I was stuck.  Well today you now have up to 10 apps that you can run on appspot or your own domain. Make sure to update your SDK.

Next question is, when are they going to launch this out of beta?  I want to start using it for business.

We’re happy to announce we’ve released some small updates to Google App Engine. Among the more significant changes:

• More apps: Want to create more than 3 applications with your App Engine account? Now you can now create up to 10!
• Time windows for Dashboard graphs: Zoom in on the data in your dashboard to get a more accurate picture of whats going on. You can zoom in to see graphs for the last 24, 12, and 6 hour periods.
• Logs export: You can now use appcfg.py to download your application’s logs in plaintext format. Use appcfg.py –help for more information on how to download your logs.
• Send email as logged in user: If you’re using the users API, you can now send email from the email address of the currently-logged-in user.

Be sure to update your SDK and check the release notes for a full list of changes. Have more changes you’d like to see with App Engine? Let us know in our Google Group!

Today Mosso, a cloud provider that runs off of Rackspace and supports lots of languages, launched CloudFS to compete with Amazon S3.

CloudFS is new, untested but a bit cheaper than Amazon Simple Storage Service (S3). This is where mainly ‘buckets’ of data of any content type can be stored and retrieved by a unique key across all resources. This is useful for image, content and media hosting and charges by the GB usually less than .15 cents per GB.

About CloudFS

ElasticFox is a pretty nice tool for managing EC2 (Amazon Elastic Compute Cloud) AMI instances AMIs (Amazon Machine Instances) from preconfigured AKIs (Amazon Kernel Image) or ARIs (Amazon Ramdisk Instances). For instance they have a AMI for fedora core 4 that is loaded with mysql and apache form amazon to get started quickly. Ultimately you end up making your own AMIs with the stuff you run on but these can get you started quickly.

It looks like this, it is built in XUL but could easily be built in AJAX or Flash or even Silverlight using APIs.

Figure 1: Shows the AMIs and your installed AMI instances.

Figure 2: Shows Available Preconfigured instances from Amazon and Others

Pretty nice little GUI to the EC2 service to help people ease into setting up and playing with cloud computing. For people running on the cloud already this is nice to have a quick web developer tool for testing, and quickly changing the dynamics of your resources with in your browser.

I think the cloud will be victorious sooner rather than later if there are great tools to beat out traditional hosting. Finally making tools that aren’t locked to a hosting company with bad applications. At least it makes this area more competitive.

This is what makes Amazon’s model so attractive. Even though it is pieces and components, services will be built on it with great interfaces, probably much better ones than can be designed by the cloud provider themselves via great apis and componentization of cloud infrastructure.

• Download ElasticFox
• Amazon post about ElasticFox

ZDNet reports that Microsoft officially dropped a beta of Live Mesh to a familiar 10,000 developers, of which many slots were taken by Microsoft employees themselves who are programmers annoyingly referred to as ’softies’.

The service or system built on another term called Horizon is their competitor to Amazon’s AWS platform and Google App Engine as well as many other emerging companies and platforms offering scalable cloud based services.

From the looks of the schematics and diagrams it appears closer to a Google App Engine hosted service rather than a componentized Amazon model.

Amazon EC2 lapped up again and added EC2 persistent storage. Previously you had to store data in SimpleDB or S3 because your EC2 instances could go away at any time due to scale down and up. This is much like how on Google App Engine you need to store everything in the BigTable Datastore, because there is really no such thing as a physical location in a cloud that is elastic. But Amazon has one upped by adding persistant storage in EC2.

If you are a developer and not excited about the ability to make applications with no worry of scale or support in infrastructure, I am just not sure if you are a developer or more worried about change. I always have these ideas that end up being based on the fast that I would need at least 3-5 servers to pull off. Unfortunately when you are smaller or just trying out an idea you dont’ want to buy infrastructure for the most extreme case (large traffic spike) and then have servers sitting at 1% utilization the rest of the time. With cloud computing and service storage, this whole problem is abstracted away. Many times this is at a much reduced cost, you are only paying for minimal usage, not what your maximum usage would be.

One big element to the cloud computing rage is also that dynamic languages will win out. If you will note in all the new cloud technologies there is a recurring theme of BigTable/MapReduce datastores (eliminiating much of relational database ways), dynamic languages (removing static typing and adding flexibility for rapid deployment and development) and the developer has alot of freedom and power to innovate.

Consider cloud computing the point at which dynamic languages such as Python (maybe Jython, IronPython, etc), Ruby, and higher order parallel languages like Erlang won out. Google App Engine uses Python, SimpleDB from Amazon is built with Erlang, etc. These languages like Python and Ruby which are slower (Ruby is much much slower than Python, Python can hang with static compiled languages) but the scalability, ActiveRecord like BigTable architecture, and rapid prototyping ORMs which were previously seen as too performance intensive or slow could become moot with the power of Amazon or Google’s infrastructure.

It is just a real interesting time in technology, it a major leap, it should be recognized. If you are a developer or infrastructure specialist, you should be diving in.

Sun is also getting into the cloud computing and storage game with project Caroline.  I have heard Microsoft will also be joining possibly with CloudDB but all that is rumors right now.

The only companies in it now are Amazon and Google is emerging. There are also some competitors like Joyent.  But I imagine there will be many companies propping up that are using Amazon or Google or other clous services as their core product.

Amazon is still far far ahead in offering componentized pieces of architecture such as S3, EC2, SQS etc.

Google App Engine is in beta and was announced today. They were supposed to release this week and they did.

What is the Google App Engine?

It is actually a bit different than I expected. I expected these services to directly compete with S3, EC2, SimpleDB and they do but as of right now you can only run Google App Engine apps on their servers. Amazon Web Services you can run on any server which makes for a much more flexible usage into existing apps. Not everything yet can be hosted in the cloud and having the access to store locally or in the cloud is key, right now Amazon has that right model for it. Amazon isn’t trying to be your authentication service or push you into other services, it is simply an extension of your domain. However Google’s offerings are very compelling here and threaten lots more markets than just amazon and salesforce but also hosting companies.

Google is looking like they are trying to take over the domain which might work but I am more for the parallel development. For instance, if you wanted to run storage on Google but computing on EC2 could you do that with Google’s setup, no. With Amazon you can swap out S3, EC2, SimpleDB for other things if you want. I like the componentized model much much better. Hopefully Google moves in the component model as well as their hosted solutions, this is very nice for upstart and getting things moving quickly though. Google does want to become the base.

On the other side, SalesForce is probably freaking out because if you use Google Apps for your business, this just adds in the element of business apps that can be run for entire small business infrastructure with Google Apps and any custom apps needed with Google App Engine. But they probably will adapt to use this within their systems, if it was more componentized and available outside Google App Engine.

I will post more on this when it has been through the wringer. As of right now Google’s service is extremely beta, limited on their servers and only available for small traffic. When they release pricing and more information we will see where the market falls for cloud based storage and computing.

A few things I really like about the new Google offerings is simplicity. They are using Python which arguably has some of the best toolkits out there for cloud development right now (boto being one of them for Amazon). They are pretty much using Pure Python and you can push up whatever libraries you want. They have Django and I will see if other templating engines like pyTenjin work up there. But also their configuration is in a simple format. Simple user formats I am fond of are Markdown and YAML. This is how you configure an app for Google App Engine, in YAML (Yet Another Markup Language):

application: myapp
version: 1
runtime: python
api_version: 1
 
handlers:
- url: /
script: home.py
 
- url: /index.html
script: home.py
 
- url: /stylesheets
static_dir: stylesheets
 
- url: /(.*\.(gif|png|jpg))
static_files: static/\1
upload: static/(.*\.(gif|png|jpg))
 
- url: /admin/.*
script: admin.py
login: admin
 
- url: /.*
script: not_found.py

However I see Amazon still being the champ here when comparing the current publicly known offerings. Hopefully Google is just testing their engines and infrastructure in a limited capacity and will open it up to componentization to run from anywhere, much like OpenSocial or other services like RESTful, XML-RPC, JSON-RPC etc. As of right now Google App Engine is too closely coupled for integration into many systems residing on servers not on Google’s environment.

I think Google’s App Engine service more closely matches the SalesForce.com type hosted SaaS rather than true cloud technologies just yet. It is a bit of a mix of both.

It sure is an interesting time seeing and participating in another game changing announcement and new pardigm and bend in the software market from local to cloud computing and storage.

• App Gallery
• Google App Engine Docs
• Google App Engine

Google has some nice setup though with these great Python libraries running for your disposal:

In addition to the Python standard library and the App Engine libraries, the runtime environment includes the following third-party libraries:

• Django 0.96.1
• WebOb 0.9
• PyYAML 3.05

They are using django templates, these are pretty nice I have taken a liking to Mako and pyTenjin but django framework is quite nice.

Here is a video walk through of the application run through. Basic Python templating app essentially in django.

Google App Engine Walkthrough

Guido Speaks

What is the web and software cloud?  

Software and data storage has always evolved, and quite rapidly in technology.  From the old punch cards, to disks, to hard drives and now a new type of storage is emerging, the cloud.

Distributed computing based on the internet, the ultimate self-healing network, is now reaching a level where services that provide distributed data storage are thriving.  I will give you three areas that these types of services are happening and why you see this area of technology is set for leaps and bounds.

• Media and Digital Asset Storage (who doesn’t use S3 in at least some projects?)
• Software Configuration Management (SCM) Distributed Source Control (Git, Mercurial, etc)
• Personal Data

Media and Digital Asset Storage

Software Configuration Management (SCM)

Personal Data

And More