Power is a micro framework which wraps Tornado to make it easy to write a web application using Tornado. Power is refined from the common components of http://poweredsites.org which is an open source web site to show a project(eg.Python, Tornado) powered sites.

The source code is coming soon, it will be opened at http://bitbucket.org/felinx/power

Install(in ubuntu)

sudo apt-get install nginx

Main configuration

# update default configuration
sudo nano /etc/nginx/nginx.conf

#
# /etc/nginx/nginx.conf
#

# user and group to run as
user       www-data www-data;

# number of nginx workers
# the same as the core numbers
worker_processes  2;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

worker_rlimit_nofile 8192;
events {
    # Number of worker connections. 2048 is a good default
    worker_connections  2048;
    use epoll;
}

http {
    upstream frontends {
        server 127.0.0.1:8888;            
        #server 127.0.0.1:8891;
        #server 127.0.0.1:8892;
        #server 127.0.0.1:8893;
        #server 127.0.0.1:8894;
    }

    server_tokens on;

    # pull in mime-types.
    include  mime.types;

    # set a default type for the rare situation that
    # nothing matches from the mimie-type include
    default_type application/octet-stream;

    access_log  /var/log/nginx/access.log;

    gzip on;
    gzip_min_length  1024;
    gzip_buffers     16 8k;   
    gzip_http_version 1.1; 
    gzip_comp_level  2;
    gzip_types       text/plain text/css application/x-javascript text/xml 
                        application/xml application/xml+rss text/javascript;
    gzip_vary on;

    sendfile     on;
    tcp_nopush   on;
    tcp_nodelay  on;

    keepalive_timeout  60;

    client_max_body_size    10m;
    client_body_buffer_size 256k;

    # proxy settings
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Scheme        $scheme;
    proxy_connect_timeout   90;
    proxy_send_timeout      90;
    proxy_read_timeout      90;
    proxy_buffers           32 8k;

    include /etc/nginx/sites-enabled/*;
}

A site's configuration(For example:poweredsites.org)

sudo nano /etc/nginx/sites-enabled/poweredsites.conf

#
# /etc/nginx/sites-enabled/poweredsites.conf
# 
server {
    listen   80;
    server_name  static.poweredsites.org;
    access_log  /var/log/nginx/poweredsites.static.log;

    location / {
        root /mnt/ebs/sites/poweredsites/poweredsites/static/;
        if ($query_string) {
            expires max;
        }
    }
}

server {
    listen   80;
    server_name  poweredsites.org;
    access_log  /var/log/nginx/poweredsites.access.log;

    location / {     
        proxy_pass http://frontends;
    }
}

server {
    listen   80;
    server_name  *.poweredsites.org;
    access_log  /var/log/nginx/poweredsites.access.log;

    # permanent redirect www.poweredsites.org to poweredsites.org
    if ($host ~* ^www\.(.*)$) {
        set $domain $1;
        rewrite ^(.*)$ http://$domain$1 permanent;
    }

    location / {     
        proxy_pass http://frontends;
    }
}

Download and install

$ curl http://downloads.mongodb.org/linux/mongodb-linux-i686-1.6.1.tgz > mongo.tgz
$ tar xzf mongo.tgz

Create a data directory

# create default data directory
$ mkdir -p /data/db/
# Grant permission for user-xxx which is the user to run mongodb process
$ chown user-xxx /data/db

Run MongoDB

$ ./mongodb-xxxxxxx/bin/mongod

You should run it with --dbpath parameter like below if data directory is not /data/db/

$ ./mongodb-xxxxxxx/bin/mongod --dbpath=/mnt/ebs/data/mongodb

Add database users

$ ./mongodb-xxxxxxx/bin/mongo
# add a admin user 
> use admin
> db.addUser("admin", "adminpassword")
# add user for poweredsites database
> use poweredsites
> db.addUser("felinx", "felinx")

# show users
> db.system.users.find()

For detail information please refer to the official documentation:

http://www.mongodb.org/display/DOCS/Quickstart+Unix http://www.mongodb.org/display/DOCS/Security+and+Authentication

Install(in ubuntu)

apt-get install mysql-server libmysqlclient15-dev

Create user and database

# login as root at first
mysql -u root -p

# create a user and set password
mysql> create user 'your-name'
mysql> upadte user set password=PASSWORD('your-password') where user='your-name';
mysql> flush privileges;

# grant user's privileges for local access
mysql> grant all privileges on *.* to your-name@"localhost" identified by "your-local-password";

# grant user's privileges for remote access(optional)
mysql> grant all privileges on *.* to your-name@"%" identified by "your-remote-passwd";
mysql> flush privileges;

# show users
mysql> select * from user;

# create database
mysql> create database your-database

Charset(UTF-8) and storage engine(INNODB)

# modify default settings
nano /etc/mysql/my.cnf

# add below settings to my.cnf
[client]
default-character-set=utf8

[mysqld]
# The default character set that will be used when a new schema or table is
# created and no character set is defined
default-character-set=utf8

# The default storage engine that will be used when create new tables
default-storage-engine=INNODB

Change default data path(Optional)

eg. change default data path from /var/lib/mysql to /mnt/ebs/data/mysql

# copy data files,copy with -p to preserve folder's attibutes
cp -r -p /var/lib/mysql/ /mnt/ebs/data

# change mysqld's apparmor, make sure data path correct everywhere.
nano /etc/apparmor.d/usr.sbin.mysqld

# modify old path to new's
#/var/lib/mysql r,
#/var/lib/mysql/** rwk,

/mnt/ebs/data/mysql r,
/mnt/ebs/data/mysql** rwk,

Restart

/etc/init.d/apparmor restart
/etc/init.d/mysql restart

Check charset setting

# execute it in mysql console or any mysql client tools
show variables like '%char%';

Remote access(Optional)

nano /etc/mysql/my.cnf
# comment bind-address in my.cnf, and grant privilages for remote access(Refer 
# to create user section)

#bind-address = 127.0.0.1

Access issue

Some guys may get ERROR 1045 (28000) when they try to login MySQL as root at first time, Error message like "Access denied for user 'root'@'localhost' (using password: YES)". You can use reserved debian-sys-maint user to login in ubuntu or debian:

mysql -u  debian-sys-maint -p
# enter password in /etc/mysql/debian.cnf
# then update root's password
mysql> upadte user set password=PASSWORD('new-password') where user='root';
mysql> flush privileges;

Here is the steps to run poweredsites project in a local ubuntu

Python2.6 ENV

Please run commands in this section to make sure all libs are ready:

# install git to sync tornado source code, and build tools etc
sudo apt-get install git-core gcc make build-essential

# python basic tools
sudo apt-get install python-setuptools python2.6-dev

# third party libs 
sudo apt-get install python-pycurl python-mysqldb

sudo easy_install -U pymongo markdown beautifulsoup
                webhelpers formencode routes decorator

# tornado
git clone git://github.com/facebook/tornado
# And go to tornado folder run
sudo python setup.py install

Database

• Please refer to How to setup MySQL and How to setup MongoDB to setup MySQL and MongoDB

• add database poweredsites and user:felinx password:felinx to mysql and mongodb,of course, you can use your username and password, then change the project configuration.

• import the schema in the repository to your mysql (poweredsites/conf/schema.sql)

Modify local Hosts

Mapping some subdomains to local machine, you can add more for testing:

sudo nano /etc/hosts

#
# /etc/hosts
#
127.0.0.1       static.poweredsites.org
127.0.0.1       www.poweredsites.org    
127.0.0.1       sites.poweredsites.org
127.0.0.1       tornado.poweredsites.org
127.0.0.1       jquery.poweredsites.org    
127.0.0.1       poweredsites.org

Server

Proxy poweredsites.org requests to localhost:8888(Apache or Nginx as your wish) Here is a Nginx exmaple, please refer to How to setup Nginx to setup nginx. The default port 80 will be conflict with apache2 if you already install apache2. If so, stop your apache2 or change apache2 listened port to another one.

Project

Yes, I hardcode something :(, I will refine that later.

• put a symbol link or copy poweredsites/poweredsites/poweredsites.conf to /mnt/ebs/conf/sites/poweredsites.conf

• Sync project root to /mnt/ebs/sites/poweredsites, then the layout of those files will like below.

  /mnt/ebs/sites/poweredsites/bin
  /mnt/ebs/sites/poweredsites/conf
  /mnt/ebs/sites/poweredsites/poweredsites
  /mnt/ebs/sites/poweredsites/poweredsites/static
  /mnt/ebs/sites/poweredsites/poweredsites/app.py
  

Run the app:

# setup mongodb index at the first time
python /mnt/ebs/sites/poweredsites/poweredsites/app.py --setup_db=1

# Run this one in normal, or run the upper one always.
python /mnt/ebs/sites/poweredsites/poweredsites/app.py

# for short
cd /mnt/ebs/sites/poweredsites/poweredsites/
python app.py

Then the app will be run at port 8888, type http://poweredsites.org in your browser to play with it. Use CONTROL + C to Quit the app.

Well done, congratulations, Have fun！

I am very proud to announce the 1.0 release of http://poweredsites.org after two week's beta online testing.

Thank Bret and Ben for the great Tornado project.

Thank those guys who have given me good feedbacks which help me a lot to improve this site.Thank every one who has submitted sites or projects to PoweredSites.

At last, I need thank James Zhang who help me setup up the Amazon instance and draft site's Terms and Privacy Policy, he also gave me some good suggestions.

Let's cheer the official 1.0 release of PoweredSites.

1. Remove some features(eg.chat, wiki) to keep poweredsites simple and easy to use, focus on listing a project or service powered sites.

2. List latest sites at home page, and move old version home page to project main page. So, it's easy to understand what is this site's main theme.

3. Add top sites list(order by page view and page rank etc.)

PowerdSites' site feeds and blog feeds are available now, you can subscribe them with your favorite reader, cheers.

PoweredSites' in-house caching system has been enabled, generally, the performance is boosted over 50 times in the server side.For example, it will only take 2ms to render the homepage from cache if cache is enabled and is not expired, or it need do more database queries and templates rendering, so it will take about 170ms.

The caching system can be used to cache

1. A whole web page
2. An UIModule
3. A normal functional in handler or UIModule.

There are three cache decorators:

1. cache.page, cache a whole web page to MonogoDB. It is just used to decorate SUPPORTED_METHODS (eg. get and post) in a handler.
2. cache.cache, cache a functional or UIModule(render functional in a UIModule) to MonogoDB.
3. cache.mem, cache a functional or UIModule to Python dict in memory. It just be used for caching some data which are seldom changed but the data are used frequently.

How to use them, let's see the cache arguments at first.

def cache(expire=7200, condition="", key="", anonymous=False):
    """Decorator which caches the value of a method in a handler or a module.

    expire: Cache will be expired time from now in seconds.
    condition: If the result of sql condition has changed, then cache expired. 
    key: The unique key of the cache identify in the DB, it will auto generate 
         one if it not be set

    cache_pre: A method which is defined in self(handler or module), 
                it always be executed before cache or get from cache.

    cache_condition: A property which is defined in self(handler or module), it
                is used to construct a complex condition.
    """
    def wrapper(func, self, *args, **kwargs):
        # caching

Then let's show the power in some examples:

• cache a whole page

  class IndexHandler(BaseHandler):
      @cache.page(expire=3600)
      def get(self):                
          self.render("index.html")
  

• just cache for anonymous user

  class IndexHandler(BaseHandler):
      @cache.page(anonymous=True)
      def get(self):                
          self.render("index.html")
  

• cache a functional

  class IndexHandler(BaseHandler):
      @cache.cache()
      def do_complex_query(self):
          # do time cosuming queries and return the values
          self.db.query("complex query 1")
          self.db.query("complex query 2")           
          return "query resluts"
  

• cache an ui_module

  class IndexModule(UIModule):
      @cache.cache()
      def render(self):
          self.render_string("index_module.html")
  

• cache to memory

  class TomemModule(UIModule):
      @cache.mem()
      def render(self):
          self.render_string("tomem_module.html")
  

• cache with condition checking

  class IndexHandler(BaseHandler):
      # The cache will expired immediately if the count(*) has changed, 
      # for example a new blog is posted, so the cache is very dynamic.
      @cache.page(condition="select count(*) from entries")
      def get(self):               
          self.render("index.html")
  

• cache with a complex condition and do some cache_pre operations

  class IndexHandler(BaseHandler):
      @property
      def cache_condition(self):
          # It will try to use this property as cache condition if no condition
          # argument. You can construct a complex condition here as your wish.
          return str("select updated from entry where id = %s" % self.entry_id)
  
      def cache_pre(self, entry_id):
          # It always will be executed before `get` or `post` and cache_condition, 
          # it has the same arguments as `get` or `post`.
          # You can do something here before try to get the cache.
  
          self.entry_id = entry_id
  
          # Do cache_pre operations
          # eg. update the click count of this entry
  
      @cache.page()
      def get(self, entry_id):            
          self.render("entry.html")
  

Please refer to the source code for more detail information.

There is an edit link beside site name in your site detail information page.

Click it, then you can re-edit your site information again, eg. modify site's logo or description.

Besides, I also update chat channel, there is a link for the author of a message, and list the latest messages at chat home page.

Fix textarea display error when submit a site or project failed.

I am very pleased to announce the beta release of http://poweredsites.org

What powered sites?

We are always curious about that when we navigating exciting sites. PoweredSites is a good site to share with others about a project or service powered sites, eg. jQuery powered sites, Torando powered sites. PoweredSites is also a good place to show your project's power if you are a project owner.

PoweredSites source code is opened to the community under Apache License V2, the source code is available at http://www.bitbucket.org/felinx/poweredsites, it's develop by python and tornado. I opened everything except the OpenID API key:), so you can setup a web site using the code easily.

Currently, twitter,facebook,friendfeed openid login are no well tested because those web sites are blocked in China. And some of features are still under developing, eg. re-edit a project or site and wiki for every project.

I hope that you will enjoy the site and the source code. I hope some guys can join me to make the site better.

Please don't hesitate to submit your projects or submit web sites to http://poweredsites.org/.

And any bug reports are welcome.