Google AppEngine is finally allowing applications to go beyond the quotas.  Mainly the 500 GB transfer limit was the killer because you hit a brick wall if your app was popular, unless you got a special quota which was the case on some apps helping to promote GAE. Now, to help highlight the cloud scale rather than look like a small beans hosting account with limits, it will go beyond the quotas. Of course,you will have to pay beyond that.

The pricing for space beyond the free quotas is fairly decent.  About the numbers that were put out earlier around .10 to .15 a GB.

Making the limits larger with a pay version is great for many reasons.  First for Google shareholders and most of all developers that want to develop real applications and services on GAE without the fear of hitting the brick wall when the quotas are up.  But now that wall is a payment.  I was hoping they would lift the quotas a bit more before they rolled in pricing, instead they lowered the quotas…

The pricing for resources beyond those free quotas is:

• $0.10 per CPU core hour. This covers the actual CPU time an application uses to process a given request, as well as the CPU used for any Datastore usage.
• $0.10 per GB bandwidth incoming, $0.12 per GB bandwidth outgoing. This covers traffic directly to/from users, traffic between the app and any external servers accessed using the URLFetch API, and data sent via the Email API.
• $0.15 per GB of data stored by the application per month.
• $0.0001 per email recipient for emails sent by the application

The big bummer in all this is the comment that they are reducing the free quotas:

App Engine remains free to get started. However, along with many performance improvements over the past ten months, we’ve learned that we overestimated our initial free quota values. Therefore, in 90 days we will be reducing the free quota resources. We believe these new levels will continue to support a reasonably efficient application serving around 5 million page views per month, completely free.

The free quota on transfer rate is currently 500GB but will go down to approx 300GB transfer.  This will probably support a 2-5m request app monthly but the old quotas were almost double that.  So it may change some economics that were being planned for GAE.  Let’s hope the free quotas only go up like gmail from now on.

Now we just need full support for full text search, which will become the Google feature benefit I believe, hopefully soon.

Great video from Google on building for scale in Google AppEngine. The cloud by nature gives you upwards scaling easily but the data structures and techniques to get data efficiently is much different than RDBMS type optimization. Therefore the more experience and information you can gain on this topic will help you in the future when terabytes of data is the norm.

The slides from the video

Google AppEngine SDK has been updated to 1.1.9 and a feature bug has been closed out by Guido that now has support for urllib and urllib2 libraries which nearly every library made with python uses as some point.  This is great that they are working to support (safely with scalability) the common libraries that Pythonistas expect.

I really hope Google AppEngine can go 1.0 soon.  After the official launch of Amazon EC2 and other services out of beta, mosso, joyent etc it is getting hard to convince people to build on it until it goes out of beta.

Hoping for two things, 1.0 of Google AppEngine and support of Django 1.0 on that release officially. It is possible to run Django 1.0.x on the engine but it is painful using all that disk space (django has lots of files and google has a 1,000 file limit).

http://code.google.com/p/googleappengine/issues/detail?id=61

Google AppEngine blog recently announced the 6 month roadmap.  I was hoping an out of beta state was to come soon but looks like it will be a gmail type beta length.  But it is still usable and more stable with high availability than anything that s small company can provide.  I hope they plan to go live to 1.0 this year, I was hoping the EC2 announcements and official launch of other cloud based offerings would drive them to do that.  It makes it much easier to justify using it for production code. I was also hoping for an update to the included django to 1.0.

With that said I digress, the roadmap looks very nice.  Jabber/XMPP support, background processesses, sending and receiving and processing inbound and outbound email.

The App Engine team has been plugging away and we’re excited about some pretty big announcements in the near future. In the meantime, we decided to refresh our App Engine roadmap for the next six months with some of the great new APIs in our pipeline:

• Support for running scheduled tasks
• Task queues for performing background processing
• Ability to receive and process incoming email
• Support for sending and receiving XMPP (Jabber) messages

As always, keep in mind that development schedules are notoriously difficult to predict, and release dates may change as work progresses. We’ll do our best to update this roadmap as our engineers continue development and keep you abreast of any changes!

Google AppEngine is alot of fun, not only is it a good excuse to use Python but it is a touch of the future and lots of possibilities for programmers and engineers from all sized businesses to use. When they opened the gates I was one of the lucky 10,000 to get in. So what did I do, I setup three apps before I even knew that was the limit.  Then I was stuck.  Well today you now have up to 10 apps that you can run on appspot or your own domain. Make sure to update your SDK.

Next question is, when are they going to launch this out of beta?  I want to start using it for business.

We’re happy to announce we’ve released some small updates to Google App Engine. Among the more significant changes:

• More apps: Want to create more than 3 applications with your App Engine account? Now you can now create up to 10!
• Time windows for Dashboard graphs: Zoom in on the data in your dashboard to get a more accurate picture of whats going on. You can zoom in to see graphs for the last 24, 12, and 6 hour periods.
• Logs export: You can now use appcfg.py to download your application’s logs in plaintext format. Use appcfg.py –help for more information on how to download your logs.
• Send email as logged in user: If you’re using the users API, you can now send email from the email address of the currently-logged-in user.

Be sure to update your SDK and check the release notes for a full list of changes. Have more changes you’d like to see with App Engine? Let us know in our Google Group!

The models and entities framework for Google Apps is pretty fun. It is very similar to django models and ruby on rails but it is even more simplified. I have been playing with Google App Engine since it came out (was in first 10,000 yay!) and it is really fun. Prototypes have never been so rapid. I wish we got more than three apps to create, I have had to repurpose a few. I have lots of ideas I will be posting here and howtos.

To create a model:

class Person(db.Model):
  first_name = db.StringProperty()
  last_name = db.StringProperty()
  hobbies = db.StringListProperty()
 
p = Person(first_name="Albert", last_name="Johnson")
p.hobbies = ["chess", "travel"]
 
p.put() # creates/updates model and inserts data

To create an Expando model
Expando models are fun because they are dynamic. You can add properties to the classes on the fly and it expands automatically.

class Person(db.Expando):
  first_name = db.StringProperty()
  last_name = db.StringProperty()
  hobbies = db.StringListProperty()
 
p = Person(first_name="Albert", last_name="Johnson")
p.hobbies = ["chess", "travel"]
 
p.chess_elo_rating = 1350
 
p.travel_countries_visited = ["Spain", "Italy", "USA", "Brazil"]
p.travel_trip_count = 13
 
p.put() # creates/updates model and inserts data

Prototyping is extremely fast with Google App Engine. Python of course is always extremely fun to do. The winds are a changin’.

[ google app engine entities docs for the DatastoreAPI ]