Cloud - Web and Service Cloud

Amazon EC2 lapped up again and added EC2 persistent storage. Previously you had to store data in SimpleDB or S3 because your EC2 instances could go away at any time due to scale down and up. This is much like how on Google App Engine you need to store everything in the BigTable Datastore, because there is really no such thing as a physical location in a cloud that is elastic. But Amazon has one upped by adding persistant storage in EC2.

If you are a developer and not excited about the ability to make applications with no worry of scale or support in infrastructure, I am just not sure if you are a developer or more worried about change. I always have these ideas that end up being based on the fast that I would need at least 3-5 servers to pull off. Unfortunately when you are smaller or just trying out an idea you dont’ want to buy infrastructure for the most extreme case (large traffic spike) and then have servers sitting at 1% utilization the rest of the time. With cloud computing and service storage, this whole problem is abstracted away. Many times this is at a much reduced cost, you are only paying for minimal usage, not what your maximum usage would be.

One big element to the cloud computing rage is also that dynamic languages will win out. If you will note in all the new cloud technologies there is a recurring theme of BigTable/MapReduce datastores (eliminiating much of relational database ways), dynamic languages (removing static typing and adding flexibility for rapid deployment and development) and the developer has alot of freedom and power to innovate.

Consider cloud computing the point at which dynamic languages such as Python (maybe Jython, IronPython, etc), Ruby, and higher order parallel languages like Erlang won out. Google App Engine uses Python, SimpleDB from Amazon is built with Erlang, etc. These languages like Python and Ruby which are slower (Ruby is much much slower than Python, Python can hang with static compiled languages) but the scalability, ActiveRecord like BigTable architecture, and rapid prototyping ORMs which were previously seen as too performance intensive or slow could become moot with the power of Amazon or Google’s infrastructure.

It is just a real interesting time in technology, it a major leap, it should be recognized. If you are a developer or infrastructure specialist, you should be diving in.

The models and entities framework for Google Apps is pretty fun. It is very similar to django models and ruby on rails but it is even more simplified. I have been playing with Google App Engine since it came out (was in first 10,000 yay!) and it is really fun. Prototypes have never been so rapid. I wish we got more than three apps to create, I have had to repurpose a few. I have lots of ideas I will be posting here and howtos.

To create a model:

class Person(db.Model):
  first_name = db.StringProperty()
  last_name = db.StringProperty()
  hobbies = db.StringListProperty()
 
p = Person(first_name="Albert", last_name="Johnson")
p.hobbies = ["chess", "travel"]
 
p.put() # creates/updates model and inserts data

To create an Expando model
Expando models are fun because they are dynamic. You can add properties to the classes on the fly and it expands automatically.

class Person(db.Expando):
  first_name = db.StringProperty()
  last_name = db.StringProperty()
  hobbies = db.StringListProperty()
 
p = Person(first_name="Albert", last_name="Johnson")
p.hobbies = ["chess", "travel"]
 
p.chess_elo_rating = 1350
 
p.travel_countries_visited = ["Spain", "Italy", "USA", "Brazil"]
p.travel_trip_count = 13
 
p.put() # creates/updates model and inserts data

Prototyping is extremely fast with Google App Engine. Python of course is always extremely fun to do. The winds are a changin’.

[ google app engine entities docs for the DatastoreAPI ]

Google App Engine is in beta and was announced today. They were supposed to release this week and they did.

What is the Google App Engine?

It is actually a bit different than I expected. I expected these services to directly compete with S3, EC2, SimpleDB and they do but as of right now you can only run Google App Engine apps on their servers. Amazon Web Services you can run on any server which makes for a much more flexible usage into existing apps. Not everything yet can be hosted in the cloud and having the access to store locally or in the cloud is key, right now Amazon has that right model for it. Amazon isn’t trying to be your authentication service or push you into other services, it is simply an extension of your domain. However Google’s offerings are very compelling here and threaten lots more markets than just amazon and salesforce but also hosting companies.

Google is looking like they are trying to take over the domain which might work but I am more for the parallel development. For instance, if you wanted to run storage on Google but computing on EC2 could you do that with Google’s setup, no. With Amazon you can swap out S3, EC2, SimpleDB for other things if you want. I like the componentized model much much better. Hopefully Google moves in the component model as well as their hosted solutions, this is very nice for upstart and getting things moving quickly though. Google does want to become the base.

On the other side, SalesForce is probably freaking out because if you use Google Apps for your business, this just adds in the element of business apps that can be run for entire small business infrastructure with Google Apps and any custom apps needed with Google App Engine. But they probably will adapt to use this within their systems, if it was more componentized and available outside Google App Engine.

I will post more on this when it has been through the wringer. As of right now Google’s service is extremely beta, limited on their servers and only available for small traffic. When they release pricing and more information we will see where the market falls for cloud based storage and computing.

A few things I really like about the new Google offerings is simplicity. They are using Python which arguably has some of the best toolkits out there for cloud development right now (boto being one of them for Amazon). They are pretty much using Pure Python and you can push up whatever libraries you want. They have Django and I will see if other templating engines like pyTenjin work up there. But also their configuration is in a simple format. Simple user formats I am fond of are Markdown and YAML. This is how you configure an app for Google App Engine, in YAML (Yet Another Markup Language):

application: myapp
version: 1
runtime: python
api_version: 1
 
handlers:
- url: /
script: home.py
 
- url: /index.html
script: home.py
 
- url: /stylesheets
static_dir: stylesheets
 
- url: /(.*\.(gif|png|jpg))
static_files: static/\1
upload: static/(.*\.(gif|png|jpg))
 
- url: /admin/.*
script: admin.py
login: admin
 
- url: /.*
script: not_found.py

However I see Amazon still being the champ here when comparing the current publicly known offerings. Hopefully Google is just testing their engines and infrastructure in a limited capacity and will open it up to componentization to run from anywhere, much like OpenSocial or other services like RESTful, XML-RPC, JSON-RPC etc. As of right now Google App Engine is too closely coupled for integration into many systems residing on servers not on Google’s environment.

I think Google’s App Engine service more closely matches the SalesForce.com type hosted SaaS rather than true cloud technologies just yet. It is a bit of a mix of both.

It sure is an interesting time seeing and participating in another game changing announcement and new pardigm and bend in the software market from local to cloud computing and storage.

• App Gallery
• Google App Engine Docs
• Google App Engine

Google has some nice setup though with these great Python libraries running for your disposal:

In addition to the Python standard library and the App Engine libraries, the runtime environment includes the following third-party libraries:

• Django 0.96.1
• WebOb 0.9
• PyYAML 3.05

They are using django templates, these are pretty nice I have taken a liking to Mako and pyTenjin but django framework is quite nice.

Here is a video walk through of the application run through. Basic Python templating app essentially in django.

Google App Engine Walkthrough

Guido Speaks