Web2py8.5plus3minus4plus24plus2minus2Full Stack Web Framework, 4th Edition Web2py Manual 4th.1

User Manual: Pdf

Open the PDF directly: View PDF PDF.
Page Count: 583 [warning: Documents this large are best viewed by clicking the View PDF Link!]

Copyright 2008-2012 by Massimo Di Pierro. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording, scanning,
or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copy-
right Act, without either the prior written permission of the Publisher, or authorization
through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc.,
222 Rosewood Drive, Danvers, MA 01923, (978)750-8400, fax (978)646-8600, or on the web at
www.copyright.com. Requests to the Copyright owner for permission should be addressed
Massimo Di Pierro
School of Computing
DePaul University
243 S Wabash Ave
Chicago, IL 60604 (USA)
Email: massimo.dipierro@gmail.com
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their
best efforts in preparing this book, they make no representations or warranties with respect
to the accuracy or completeness of the contents of this book and specifically disclaim any
implied warranties of merchantability or fitness for a particular purpose. No warranty
may be created ore extended by sales representatives or written sales materials. The advice
and strategies contained herein may not be suitable for your situation. You should consult
with a professional where appropriate. Neither the publisher nor author shall be liable for
any loss of profit or any other commercial damages, including but not limited to special,
incidental, consequential, or other damages.
Library of Congress Cataloging-in-Publication Data:
ISBN: 978-0-578-09793-0
Build Date: December 9,2011
to my family
1Introduction 21
1.1Principles.............................. 23
1.2Webframeworks ......................... 24
1.3Model-View-Controller . . . . . . . . . . . . . . . . . . . . . . 26
1.4Why web2py............................ 29
1.5Security............................... 31
1.6Inthebox ............................. 34
1.7License............................... 35
1.8Acknowledgments ........................ 37
1.9Aboutthisbook.......................... 38
1.10 Elementsofstyle ......................... 40
2The Python language 43
2.1AboutPython ........................... 43
2.2Startingup............................. 44
2.3help,dir .............................. 45
2.4Types................................ 46
2.4.1str ............................. 46
2.4.2list ............................. 47
2.4.3tuple ............................ 49
2.4.4dict ............................. 50
2.5Aboutindentation ........................ 51
2.6for...in ............................... 52
2.7while ................................ 53
2.8if...elif...else .......................... 53
2.9try...except...else...finally .................. 54
2.10 def...return ............................ 56
2.10.1lambda ........................... 58
2.11 class ................................ 60
2.12 Special attributes, methods and operators . . . . . . . . . . . 62
2.13 Fileinput/output......................... 63
2.14 exec,eval .............................. 64
2.15 import ................................ 65
2.15.1os .............................. 65
2.15.2sys ............................. 66
2.15.3datetime .......................... 67
2.15.4time ............................. 67
2.15.5cPickle ........................... 68
3Overview 69
3.1Startup ............................... 69
3.2Sayhello .............................. 73
3.3Letscount............................. 78
3.4Saymyname ........................... 79
3.5Postbacks.............................. 81
3.6Animageblog........................... 84
3.7AddingCRUD........................... 98
3.8Adding Authentication . . . . . . . . . . . . . . . . . . . . . . 99
3.8.1Addinggrids .......................102
3.10 Awiki ...............................104
3.10.1On date,datetime and time format...........115
3.11 More on admin ..........................116
3.11.1site .............................116
3.11.2about ............................119
3.11.3edit .............................119
3.11.4errors ............................122
3.11.5Mercurial ..........................126
3.11.6Admin wizard (experimental) . . . . . . . . . . . . . 127
3.11.7Configuring admin ....................129
3.12 More on appadmin ........................130
4The core 133
4.3Dispatching ............................139
4.4Libraries ..............................143
4.6API .................................151
4.6.1Accessing the API from Python modules . . . . . . . 152
4.7request ...............................155
4.8response ...............................160
4.9session ...............................164
4.10 cache ................................166
4.11 URL ..................................169
4.11.1Absoluteurls .......................172
4.11.2Digitally signed urls . . . . . . . . . . . . . . . . . . . 172
4.12 HTTP and redirect .........................173
4.13 Tand Internationalization . . . . . . . . . . . . . . . . . . . . 175
4.14 Cookies ...............................178
4.15 Application init ..........................179
4.16 URLrewrite ............................179
4.16.1Parameter-based system . . . . . . . . . . . . . . . . . 180
4.16.2Pattern-based system . . . . . . . . . . . . . . . . . . 182
4.17 Routesonerror ..........................186
4.18 Running tasks in the background . . . . . . . . . . . . . . . . 187
4.18.1Cron ............................188
4.18.2Homemade task queues . . . . . . . . . . . . . . . . . 191
4.18.3Scheduler (experimental) . . . . . . . . . . . . . . . . 192
4.19 Thirdpartymodules .......................195
4.20 Execution environment . . . . . . . . . . . . . . . . . . . . . . 197
4.21 Cooperation ............................198
4.22 Logging...............................199
4.23 WSGI ................................200
4.23.1External middleware . . . . . . . . . . . . . . . . . . . 201
4.23.2Internal middleware . . . . . . . . . . . . . . . . . . . 201
4.23.3Calling WSGI applications ...............202
5The views 203
5.1Basicsyntax ............................205
5.1.1for...in ..........................205
5.1.2while ............................206
5.1.3if...elif...else .....................206
5.1.4try...except...else...finally .............207
5.1.5def...return ........................208
5.2.1XML .............................211
5.2.2Built-inhelpers ......................212
5.3BEAUTIFY ...............................229
5.4Server-side DOM andparsing..................229
5.4.1elements ..........................229
5.4.2components .........................231
5.4.3parent ...........................231
5.4.4flatten ...........................231
5.5.1Default page layout . . . . . . . . . . . . . . . . . . . 236
5.5.2Customizing the default layout . . . . . . . . . . . . . 240
5.5.3Mobile development . . . . . . . . . . . . . . . . . . . 240
5.6Functionsinviews ........................241
5.7Blocksinviews ..........................242
6The database abstraction layer 245
6.1Dependencies ...........................245
6.2Connectionstrings ........................247
6.2.1Connection pooling . . . . . . . . . . . . . . . . . . . 248
6.2.2Connection failures . . . . . . . . . . . . . . . . . . . . 249
6.2.3Replicated databases . . . . . . . . . . . . . . . . . . . 249
6.4DAL,Table,Field ..........................250
6.5Record representation . . . . . . . . . . . . . . . . . . . . . . 251
6.7Fixing broken migrations . . . . . . . . . . . . . . . . . . . . 259
6.8insert ................................260
6.9commit and rollback ........................261
6.10 Rawsql...............................261
6.10.1Timingqueries ......................262
6.10.2executesql .........................262
6.10.3_lastsql ..........................262
6.11 drop .................................263
6.12 Indexes...............................263
6.13 Legacy databases and keyed tables . . . . . . . . . . . . . . . 263
6.14 Distributed transaction . . . . . . . . . . . . . . . . . . . . . . 264
6.15 Manualuploads..........................265
6.16 Query,Set,Rows ...........................266
6.17 select ................................266
6.17.2Fetching a Row .......................269
6.17.3Recursive selects.....................270
6.17.4Serializing Rows inviews.................271
6.17.5orderby,groupby,limitby,distinct ...........273
6.17.7count,isempty,delete,update ..............275
6.17.8Expressions ........................276
6.17.9update_record .......................277
6.17.10 first and last .......................277
6.17.11 as_dict and as_list ....................277
6.17.12 find,exclude,sort .....................278
6.18 Othermethods ..........................279
6.18.1update_or_insert .....................279
6.18.2validate_and_insert,validate_and_update .......280
6.18.3smart_query (experimental) . . . . . . . . . . . . . . . 280
6.19 Computedelds .........................281
6.20 Virtualelds............................281
6.20.1Old style virtual fields . . . . . . . . . . . . . . . . . . 282
6.20.2New style virtual fields (experimental) . . . . . . . . 284
6.21 Onetomanyrelation.......................285
6.21.3Grouping and counting . . . . . . . . . . . . . . . . . 288
6.22 Manytomany...........................289
6.23 Many to many, list:<type>, and contains ...........290
6.24 Otheroperators ..........................292
6.24.1like,startswith,contains,upper,lower .........292
6.24.2year,month,day,hour,minutes,seconds .........293
6.24.3belongs ...........................294
6.24.4sum,min,max and len ...................294
6.24.6Default values with coalesce and coalesce_zero ...295
6.25 Generatingrawsql ........................296
6.26 Exporting and importing data . . . . . . . . . . . . . . . . . . 296
6.26.1CSV (one Table at a time) . . . . . . . . . . . . . . . . 297
6.26.2CSV (all tables at once) . . . . . . . . . . . . . . . . . 297
6.26.3CSV and remote database synchronization . . . . . . 298
6.26.4HTML and XML (one Table at a time) . . . . . . . . . 300
6.26.5Data representation . . . . . . . . . . . . . . . . . . . 301
6.27 Cachingselects ..........................302
6.28 Self-Reference and aliases . . . . . . . . . . . . . . . . . . . . 303
6.29 Advancedfeatures ........................304
6.29.1Tableinheritance .....................304
6.29.2Common fields and multi-tenancy . . . . . . . . . . . 305
6.29.4Custom Field types (experimental) . . . . . . . . . . 307
6.29.5Using DAL without define tables . . . . . . . . . . . 307
6.29.6Copy data from one db into another . . . . . . . . . . 308
6.29.7Note on new DAL and adapters . . . . . . . . . . . . 309
6.29.8Gotchas ..........................312
7Forms and validators 315
7.1FORM .................................316
7.1.1The process and validate methods...........320
7.1.2Hiddenelds .......................321
7.1.3keepvalues .........................322
7.1.4onvalidation ........................323
7.1.5Detect record change . . . . . . . . . . . . . . . . . . . 323
7.1.6Forms and redirection . . . . . . . . . . . . . . . . . . 324
7.1.7Multiple forms per page . . . . . . . . . . . . . . . . . 325
7.2SQLFORM ...............................326
7.2.1SQLFORM and insert/update/delete ...........332
7.2.2SQLFORM inHTML.....................333
7.2.3SQLFORM anduploads ...................334
7.2.4Storing the original filename . . . . . . . . . . . . . . 337
7.2.5autodelete .........................338
7.2.6Links to referencing records . . . . . . . . . . . . . . . 338
7.2.7Pre-populating the form . . . . . . . . . . . . . . . . . 340
7.2.8Adding extra form elements to SQLFORM ........340
7.2.9SQLFORM without database IO . . . . . . . . . . . . . . 341
7.3SQLFORM.factory ..........................342
7.3.1One form for multiple tables . . . . . . . . . . . . . . 343
7.4.1Settings ..........................345
7.4.4Recordversioning ....................351
7.5.1CSSconventions .....................354
7.5.2Hideerrors ........................354
7.6Validators .............................355
7.6.1Validators .........................357
7.6.2Database validators . . . . . . . . . . . . . . . . . . . 368
7.6.3Customvalidators ....................370
7.6.4Validators with dependencies . . . . . . . . . . . . . . 372
7.7.1Autocomplete widget . . . . . . . . . . . . . . . . . . 374
7.8SQLFORM.grid and SQLFORM.smartgrid (experimental) . . . . . . 375
8Email and SMS 383
8.1.1Configuring email for Google App Engine . . . . . . 384
8.1.2x509 and PGP Encryption . . . . . . . . . . . . . . . . 384
8.2Sendingemails ..........................384
8.2.3Combining text and HTML emails . . . . . . . . . . . 385
8.2.4cc and bcc emails.....................386
8.2.6Multiple attachments . . . . . . . . . . . . . . . . . . . 386
8.4Using the template system to generate messages . . . . . . . 387
8.5Sending messages using a background task . . . . . . . . . . 388
9Access Control 391
9.1.1Restrictions on registration . . . . . . . . . . . . . . . 396
9.1.2Integration with OpenID, Facebook, etc. . . . . . . . 397
9.1.3CAPTCHA and reCAPTCHA . . . . . . . . . . . . . . 399
9.1.4Customizing Auth .....................400
9.1.5Renaming Auth tables ..................402
9.1.6Other login methods and login forms . . . . . . . . . 402
9.2Mail and Auth ...........................409
9.3Authorization ...........................410
9.3.2Combining requirements . . . . . . . . . . . . . . . . 413
9.3.3Authorization and CRUD . . . . . . . . . . . . . . . . 414
9.3.4Authorization and downloads . . . . . . . . . . . . . 415
9.3.5Access Control and Basic Authentication . . . . . . . 416
9.3.6Manual Authentication . . . . . . . . . . . . . . . . . 416
9.3.7Settings and messages . . . . . . . . . . . . . . . . . . 417
9.4Central Authentication Service . . . . . . . . . . . . . . . . . 423
9.4.1Using web2py to authorize non-web2py apps . . . . 425
10 Services 427
10.1Rendering a dictionary . . . . . . . . . . . . . . . . . . . . . . 427
10.1.1HTML, XML, and JSON . . . . . . . . . . . . . . . . . 428
10.1.3Rendering Rows ......................430
10.2Remote procedure calls . . . . . . . . . . . . . . . . . . . . . 434
10.2.2JSONRPC .........................438
10.2.3JSONRPC and Pyjamas . . . . . . . . . . . . . . . . . 438
10.3Low level API and other recipes . . . . . . . . . . . . . . . . 445
10.3.2PyRTF ...........................446
10.3.3ReportLabandPDF ...................447
10.4RestfulWebServices .......................448
10.4.1parse_as_rest (experimental) . . . . . . . . . . . . . . 451
10.4.2smart_query (experimental) . . . . . . . . . . . . . . . 456
10.4.3AccessControl ......................457
10.5Services and Authentication . . . . . . . . . . . . . . . . . . . 457
11 jQuery and Ajax 459
11.2jQueryeffects ...........................463
11.2.1Conditional fields in forms . . . . . . . . . . . . . . . 467
11.2.2Confirmation on delete . . . . . . . . . . . . . . . . . 469
11.3The ajax function.........................470
11.3.2Auto-completion .....................471
11.3.3Ajax form submission . . . . . . . . . . . . . . . . . . 474
12 Components and plugins 479
12.1.1Client-Server component communications . . . . . . 485
12.2.1Componentplugins ...................491
12.2.3Layoutplugins ......................494
12.3plugin_wiki .............................495
12.3.1MARKMIN syntax.......................498
12.3.3Specialpages .......................501
12.3.4Configuring plugin_wiki . . . . . . . . . . . . . . . . 503
12.3.5Currentwidgets .....................503
12.3.6Extendingwidgets ....................510
13 Deployment recipes 513
13.0.7anyserver.py ........................516
13.1LinuxandUnix ..........................517
13.1.1One step production deployment . . . . . . . . . . . 517
13.1.2Apachesetup .......................517
13.1.5mod_proxy ........................523
13.1.6Start as Linux daemon . . . . . . . . . . . . . . . . . . 525
13.1.8Shared hosting with mod_python . . . . . . . . . . . 527
13.1.9Cherokee with FastCGI . . . . . . . . . . . . . . . . . 528
13.1.10 Postgresql .........................530
13.2.1Apache and mod_wsgi . . . . . . . . . . . . . . . . . 531
13.2.2Start as Windows service . . . . . . . . . . . . . . . . 534
13.3Securing sessions and admin ..................535
13.4Efficiency and scalability . . . . . . . . . . . . . . . . . . . . . 536
13.4.2Sessions in database . . . . . . . . . . . . . . . . . . . 538
13.4.3HAProxy a high availability load balancer . . . . . . 538
13.4.4Cleaning up sessions . . . . . . . . . . . . . . . . . . . 540
13.4.5Uploading files in database . . . . . . . . . . . . . . . 541
13.4.8Sessions in memcache . . . . . . . . . . . . . . . . . . 543
13.4.9CachingwithRedis ...................544
13.4.10 Removing applications . . . . . . . . . . . . . . . . . . 544
13.4.11 Using replicated databases . . . . . . . . . . . . . . . 544
13.5Deploying on Google App Engine . . . . . . . . . . . . . . . 545
13.5.2Running and deployment . . . . . . . . . . . . . . . . 548
13.5.3Configuring the handler . . . . . . . . . . . . . . . . . 550
13.5.4Avoid the filesystem . . . . . . . . . . . . . . . . . . . 550
13.5.7GAEandhttps ......................552
14 Other recipes 555
14.2How to distribute your applications as binaries . . . . . . . 555
14.3Building a minimalist web2py..................556
14.4Fetching an external URL . . . . . . . . . . . . . . . . . . . . 557
14.5Prettydates ............................558
14.7Pagination .............................558
14.8httpserver.log and the Log File Format . . . . . . . . . . . . 560
14.9Populating database with dummy data . . . . . . . . . . . . 561
14.10Accepting credit card payments . . . . . . . . . . . . . . . . . 561
14.10.2Paypal ...........................564
14.13Streaming virtual files . . . . . . . . . . . . . . . . . . . . . . 568
Bibliography 571
web2py was launched in 2007 and now, after four years of continuous
development, we have reached a very much needed fourth edition of this
book. During this time, web2py has managed to win the affection of thousands
of knowledgeable users and more than one hundred developers. Our
collective effort has created one of the most full-featured Open Source Web
Frameworks in existence.
I originally started web2py as a teaching tool because, I believe, the ability to
build high quality web applications is of critical importance for the growth
of a free and open society. It prevents the biggest players from monopolizing
the flow of information. This motivation continues to be valid and it is even
more important today.
In general, the purpose of any web framework is to make web development
easier, faster and prevent developers from making mistakes, especially in
matters related to security. In web2py we address these issues with our three
main goals:
Ease of use is the primary goal for web2py. For us, this means reducing the
learning and deployment time. This is why web2py is a full-stack framework
without dependencies. It requires no installation and has no configuration
files. Everything works out of the box, including a web server, database and a
web-based IDE that gives access to all the main features. The API includes
just 12 core objects, which are easy to work with and memorize. It can
interoperate with most web servers, database and all Python libraries.
Faster development is the secondary goal. Every function of web2py has a
20 web2py full-stack web framework,4th edition
default behavior (which can be overridden). For example, as soon as you have
specified your data models, you will have access to a web-based database
administration panel. web2py also generates automatically forms for your
data and it allows you to easily expose the data in HTML, XML, JSON, RSS,
Security is at the heart of web2py, and our goal here is to lock everything
down to keep your systems and data safe. Therefore, our database layer
eliminates SQL Injections. The template language prevents Cross Site
Scripting vulnerabilities. The forms generated by web2py provide field
validation and block Cross Site Request Forgeries. Passwords are always
stored hashed. Sessions are stored server-side by default to prevent Cookie
Tampering and session cookies are uuid to prevent Cookie Stealing.
web2py has always been built from the user perspective and is constantly
optimized internally to become faster and leaner, whilst always maintaining
backward compatibility.
web2py is free for you to use. If you benefit from it, we hope you will feel a
little more like contributing back to society in whatever form you choose.
In 2011 InfoWorld magazine reviewed six of the most popular full-stack
Python based web frameworks and raked web2py highest. Also in 2011,
web2py won the Bossie Award for best Open Source Development Software.
web2py [1] is a free, open-source web framework for agile development
of secure database-driven web applications; it is written in Python [2]
and programmable in Python. web2py is a full-stack framework, meaning
that it contains all the components you need to build fully functional web
applications. web2py is designed to guide a web developer to follow good
software engineering practices, such as using the Model View Controller
(MVC) pattern. web2py separates the data representation (the model) from
the data presentation (the view) and also from the application logic and
workflow (the controller). web2py provides libraries to help the developer
design, implement, and test each of these three parts separately, and
makes them work together. web2py is built for security. This means that
it automatically addresses many of the issues that can lead to security
vulnerabilities, by following well established practices. For example, it
validates all input (to prevent injections), escapes all output (to prevent
cross-site scripting), renames uploaded files (to prevent directory traversal
attacks). web2py leaves little choice to application developers in matters
related to security. web2py includes a Database Abstraction Layer (DAL)
that writes SQL [3] dynamically so that you, the developer, don’t have
to. The DAL knows how to generate SQL transparently for SQLite [4],
MySQL [6], PostgreSQL [5], MSSQL [7], FireBird [8], Oracle [9], IBM DB2[10],
Informix [11], and Ingres [12]. The DAL can also generate function calls for
the Google Datastore when running on the Google App Engine (GAE) [13].
22 web2py full-stack web framework,4th edition
Experimentally we support more databases. Please check on the web2py
web site and mailing list for more recent adapters. Once one or more
database tables are defined, web2py also generates a fully functional web-
based database administration interface to access the database and the
tables. web2py differs from other web frameworks in that it is the only
framework to fully embrace the Web 2.0paradigm, where the web is the
computer. In fact, web2py does not require installation or configuration;
it runs on any architecture that can run Python (Windows, Windows CE,
Mac OS X, iOS, and Unix/Linux), and the development, deployment, and
maintenance phases for the applications can be done via a local or remote
web interface. web2py runs with CPython (the C implementation) and
Jython (the Java implementation), on Python versions 2.4,2.5,2.6, and 2.7,
although "officially" it only supports 2.5so that we can guarantee backward
compatibility for applications. web2py provides a ticketing system. If an
error occurs, a ticket is issued to the user, and the error is logged for the
administrator. web2py is open source and released under the LGPL version
Another feature of web2py is that we, its developers, commit to maintain
backward compatibility in future versions. We have done so since the first
release of web2py in October, 2007. New features have been added and bugs
have been fixed, but if a program worked with web2py 1.0, that program will
still work today.
Here are some examples of web2py statements that illustrate its power and
simplicity. The following code:
1db.define_table('person',Field('name'), Field('image','upload'))
creates a database table called "person" with two fields: "name", a string; and
"image", something that needs to be uploaded (the actual image). If the table
already exists but does not match this definition, it is altered appropriately.
Given the table defined above, the following code:
1form = crud.create(db.person)
creates an insert form for this table that allows users to upload images. It
also validates a submitted form, renames the uploaded image in a secure
introduction 23
way, stores the image in a file, inserts the corresponding record in the
database, prevents double submission, and eventually modifies the form
itself by adding error messages if the data submitted by the user does not
pass validation.
The following code:
2def f(): ....
prevents visitors from accessing the function funless the visitor is a member
of a group whose members have permissions to "read" records of table
"person". If the visitor is not logged in, he gets directed to a login page
(provided by default by web2py).
The following code embeds a page component.
1{{=LOAD('other_controller','function.load',ajax=True, ajax_trap=True)}}
This instructs web2py to load in a view the content generated by the other
controller function (this works with any function). It loads the content via
Ajax, embeds it into the current page (using the current layout, not the layout
of the other_controller function), and traps all forms contained in the loaded
content so that they are also submitted via Ajax without reloading the page.
It can also LOAD content from non-web2py applications.
The LOAD helper allows very modular design of applications; it is discussed
in some detail in the last chapter of this book.
Python programming typically follows these basic principles:
Don’t repeat yourself (DRY).
There should be only one way of doing things.
Explicit is better than implicit.
web2py fully embraces the first two principles by forcing the developer to
use sound software engineering practices that discourage repetition of code.
24 web2py full-stack web framework,4th edition
web2py guides the developer through almost all the tasks common in web
application development (creating and processing forms, managing sessions,
cookies, errors, etc.).
web2py differs from other frameworks with regard to the third principle,
which sometimes conflicts with the other two. In particular, web2py does
not import user applications, but executes them in a predefined context. This
context exposes the Python keywords, as well as the web2py keywords.
To some this may appear as magic, but it should not. Simply, in practice,
some modules are already imported without you doing so. web2py is
trying to avoid the annoying characteristic of other frameworks that force
the developer to import the same modules at the top of every model and
controller. web2py, by importing its own modules, saves time and prevents
mistakes, thus following the spirit of "don’t repeat yourself" and "there
should be only one way of doing things".
If the developer wishes to use other Python modules or third-party modules,
those modules must be imported explicitly, as in any other Python program.
1.2Web frameworks
At its most fundamental level, a web application consists of a set of programs
(or functions) that are executed when the corresponding URL is visited. The
output of the program is returned to the visitor and rendered by the browser.
The purpose of web frameworks is to allow developers to build new apps
quickly, easily and without mistakes. This is done by providing APIs and
tools that reduce and simplify the amount of coding that is required.
The two classic approaches for developing web applications are:
Generating HTML [14,15] programmatically.
Embedding code into HTML pages.
The first model is the one that was followed, for example, by early CGI
scripts. The second model is followed, for example, by PHP [16] (where the
code is in PHP, a C-like language), ASP (where the code is in Visual Basic),
introduction 25
and JSP (where the code is in Java).
Here is an example of a PHP program that, when executed, retrieves data
from a database and returns an HTML page showing the selected records:
3@mysql_select_db(database) or die( "Unable to select database");
4$query="SELECT *FROM contacts";
8while ($i < mysql_numrows($result)) {
10 $phone=mysql_result($result,$i,"phone");
11 echo "<b>$name</b><br>Phone:$phone<br /><br /><hr /><br />";
12 $i++;
13 }
14 ?></body></html>
The problem with this approach is that code is embedded into HTML, but
the very same code also needs to generate additional HTML and to generate
SQL statements to query the database, entangling multiple layers of the
application and making it difficult to read and maintain. The situation is
even worse for Ajax applications, and the complexity grows with the number
of pages (files) that make up the application.
The functionality of the above example can be expressed in web2py with two
lines of Python code:
1def index():
2return HTML(BODY(H1('Records'), db().select(db.contacts.ALL)))
In this simple example, the HTML page structure is represented
programmatically by the HTML,BODY, and H1 objects; the database db is queried
by the select command; finally, everything is serialized into HTML. Notice
that db is not a keyword but a user defined variable. We will use this name
consistently to refer to a database connection to avoid confusion.
Web frameworks are typically categorized as one of two types: A "glued"
framework is built by assembling (gluing together) several third-party
components. A "full-stack" framework is built by creating components
designed specifically to be tightly integrated and work together. web2py is a
26 web2py full-stack web framework,4th edition
full-stack framework. Almost all of its components are built from scratch and
are designed to work together, but they function just as well outside of the
complete web2py framework. For example, the Database Abstraction Layer
(DAL) or the template language can be used independently of the web2py
framework by importing gluon.dal or gluon.template into your own Python
applications. gluon is the name of the web2py module that contains system
libraries. Some web2py libraries, such as building and processing forms from
database tables, have dependencies on other portions of web2py. web2py
can also work with third-party Python libraries, including other template
languages and DALs, but they will not be as tightly integrated as the original
web2py encourages the developer to separate data representation (the
model), data presentation (the view) and the application workflow (the
controller). Let’s consider again the previous example and see how to build
a web2py application around it. Here is an example of the web2py MVC edit
The typical workflow of a request in web2py is described in the following
introduction 27
In the diagram:
The Server can be the web2py built-in web server or a third-party server,
such as Apache. The Server handles multi-threading.
"main" is the main WSGI application. It performs all common tasks and
wraps user applications. It deals with cookies, sessions, transactions, URL
routing and reverse routing, and dispatching.
It can serve and stream static files if the web server is not doing it already.
The Models, Views and Controller components make up the user
Multiple applications can be hosted in the same web2py instance.
The dashed arrows represent communication with the database engine(s).
The database queries can be written in raw SQL (discouraged) or by using
the web2py Database Abstraction Layer (recommended), so that web2py
application code is not dependent on the specific database engine.
The dispatcher maps the requested URL to a function call in the controller.
The output of the function can be a string or a dictionary of symbols (a
hash table). The data in the dictionary is rendered by a view. If the visitor
requests an HTML page (the default), the dictionary is rendered into an
HTML page. If the visitor requests the same page in XML, web2py tries
28 web2py full-stack web framework,4th edition
to find a view that can render the dictionary in XML. The developer can
create views to render pages in any of the already supported protocols
(HTML, XML, JSON, RSS, CSV, RTF) or in additional custom protocols.
All calls are wrapped into a transaction, and any uncaught exception
causes the transaction to be rolled back. If the request succeeds, the
transaction is committed.
• web2py also handles sessions and session cookies automatically, and when
a transaction is committed, the session is also stored, unless specified
It is possible to register recurrent tasks (via cron) to run at scheduled times
and/or after the completion of certain actions. In this way it is possible to
run long and compute-intensive tasks in the background without slowing
down navigation.
Here is a minimal and complete MVC application, consisting of three files:
"db.py" is the model:
1db = DAL('sqlite://storage.sqlite')
It connects to the database (in this example a SQLite database stored in the
storage.sqlite file) and defines a table called contact. If the table does not
exist, web2py creates it and, transparently and in the background, generates
SQL code in the appropriate SQL dialect for the specific database engine
used. The developer can see the generated SQL but does not need to change
the code if the database back-end, which defaults to SQLite, is replaced with
MySQL, PostgreSQL, MSSQL, FireBird, Oracle, DB2, Informix, Interbase,
Ingres, and the Google App Engine (both SQL and NoSQL).
Once a table is defined and created, web2py also generates a fully functional
web-based database administration interface, called appadmin, to access the
database and the tables.
"default.py" is the controller:
1def contacts():
introduction 29
2grid=SQLFORM.grid(db.contact, user_signature=False)
3return locals()
In web2py, URLs are mapped to Python modules and function calls. In this
case, the controller contains a single function (or "action") called contacts. An
action may return a string (the returned web page) or a Python dictionary
(a set of key:value pairs) or the set of local variables (as in this example). If
the function returns a dictionary, it is passed to a view with the same name
as the controller/function, which in turn renders the page. In this example,
the function contacts generates a select/search/create/update/delete grid
for table db.contact and returns the grid to the view.
"default/contacts.html" is the view:
1{{extend 'layout.html'}}
2<h1>Manage My Contacts</h1>
This view is called automatically by web2py after the associated controller
function (action) is executed. The purpose of this view is to render the
variables in the returned dictionary (in our case grid) into HTML. The view
file is written in HTML, but it embeds Python code delimited by the special
{{ and }} delimiters. This is quite different from the PHP code example,
because the only code embedded into the HTML is "presentation layer" code.
The "layout.html" file referenced at the top of the view is provided by web2py
and constitutes the basic layout for all web2py applications. The layout file
can easily be modified or replaced.
1.4Why web2py
web2py is one of many web application frameworks, but it has compelling
and unique features. web2py was originally developed as a teaching tool,
with the following primary motivations:
Easy for users to learn server-side web development without
compromising functionality. For this reason, web2py requires no
installation and no configuration, has no dependencies (except for the
30 web2py full-stack web framework,4th edition
source code distribution, which requires Python 2.5and its standard
library modules), and exposes most of its functionality via a Web browser
• web2py has been stable from day one because it follows a top-down
design; i.e., its API was designed before it was implemented. Even as
new functionality has been added, web2py has never broken backwards
compatibility, and it will not break compatibility when additional
functionality is added in the future.
• web2py proactively addresses the most important security issues which
plague many modern web applications, as determined by OWASP [19]
• web2py is lightweight. Its core libraries, including the Database
Abstraction Layer, the template language, and all the helpers amount to
1.4MB. The entire source code including sample applications and images
amounts to 10.4MB.
• web2py has a small footprint and is very fast. It uses the Rocket [22]
WSGI web server developed By Timothy Farrell. It is 30% faster than
Apache with mod_proxy. Our tests also indicate that, on an average PC,
it serves an average dynamic page without database access in about 10ms.
The DAL has very low overhead, typically less than 3%.
• web2py uses Python syntax for models, controllers, and views, but does
not import models and controllers (as all the other Python frameworks
do) - instead it executes them. This means that apps can be installed,
uninstalled, and modified without having to restart the web server (even
in production), and different apps can cohexist without their modules
interfering with one another.
• web2py uses a Database Abstraction Layer (DAL) instead of an Object
Relational Mapper (ORM). From a conceptual point of view, this means
that different database tables are mapped into different instances of one
Table class and not into different classes, while records are mapped into
instances of one Row class, not into instances of the corresponding table
class. From a practical point of view, it means that SQL syntax maps
introduction 31
almost one-to-one into DAL syntax, and there is no complex metaclass
programming going on under the hood as in popular ORMs, which would
add latency.
WSGI [17,18] (Web Server Gateway Interface) is an emerging Python
standard for communication between a web server and Python applications).
Here is a screenshot of the main web2py admin interface:
The Open Web Application Security Project [19] (OWASP) is a free and open
worldwide community focused on improving the security of application
OWASP has listed the top ten security issues that put web applications at
risk. That list is reproduced here, along with a description of how each issue
is addressed by web2py:
"Cross Site Scripting (XSS): XSS flaws occur whenever an application takes
user supplied data and sends it to a web browser without first validating
or encoding that content. XSS allows attackers to execute scripts in the
victim’s browser which can hijack user sessions, deface web sites, possibly
introduce worms, etc." web2py, by default, escapes all variables rendered in the
view, preventing XSS.
"Injection Flaws: Injection flaws, particularly SQL injection, are common
in web applications. Injection occurs when user-supplied data is sent to
32 web2py full-stack web framework,4th edition
an interpreter as part of a command or query. The attacker’s hostile data
tricks the interpreter into executing unintended commands or changing
data." web2py includes a Database Abstraction Layer that makes SQL injection
impossible. Normally, SQL statements are not written by the developer. Instead,
SQL is generated dynamically by the DAL, ensuring that all inserted data is
properly escaped.
"Malicious File Execution: Code vulnerable to remote file inclusion
(RFI) allows attackers to include hostile code and data, resulting in
devastating attacks, such as total server compromise." web2py allows only
exposed functions to be executed, preventing malicious file execution. Imported
functions are never exposed; only actions are exposed. web2py uses a Web-based
administration interface which makes it very easy to keep track of what is exposed
and what is not.
"Insecure Direct Object Reference: A direct object reference occurs when a
developer exposes a reference to an internal implementation object, such
as a file, directory, database record, or key, as a URL or form parameter.
Attackers can manipulate those references to access other objects without
authorization." web2py does not expose any internal objects; moreover, web2py
validates all URLs, thus preventing directory traversal attacks. web2py also
provides a simple mechanism to create forms that automatically validate all input
"Cross Site Request Forgery (CSRF): A CSRF attack forces a logged-on
victim’s browser to send a pre-authenticated request to a vulnerable web
application, which then forces the victim’s browser to perform a hostile
action to the benefit of the attacker. CSRF can be as powerful as the web
application that it attacks." web2py prevents CSRF as well as accidental double
submission of forms by assigning a one-time random token to each form. Moreover
web2py uses UUID for session cookie.
"Information Leakage and Improper Error Handling: Applications
can unintentionally leak information about their configuration, internal
workings, or violate privacy through a variety of application problems.
Attackers use this weakness to steal sensitive data, or conduct more
serious attacks." web2py includes a ticketing system. No error can result in
introduction 33
code being exposed to the users. All errors are logged and a ticket is issued to the
user that allows error tracking. But errors and source code are accessible only to
the administrator.
"Broken Authentication and Session Management: Account credentials
and session tokens are often not properly protected. Attackers
compromise passwords, keys, or authentication tokens to assume other
users’ identities." web2py provides a built-in mechanism for administrator
authentication, and it manages sessions independently for each application. The
administrative interface also forces the use of secure session cookies when the client
is not "localhost". For applications, it includes a powerful Role Based Access
Control API.
"Insecure Cryptographic Storage: Web applications rarely use
cryptographic functions properly to protect data and credentials.
Attackers use weakly protected data to conduct identity theft and other
crimes, such as credit card fraud." web2py uses the MD5or the HMAC+SHA-
512 hash algorithms to protect stored passwords. Other algorithms are also
"Insecure Communications: Applications frequently fail to encrypt
network traffic when it is necessary to protect sensitive communications."
web2py includes the SSL-enabled [21] Rocket WSGI server, but it can also use
Apache or Lighttpd and mod_ssl to provide SSL encryption of communications.
"Failure to Restrict URL Access: Frequently an application only protects
sensitive functionality by preventing the display of links or URLs to
unauthorized users. Attackers can use this weakness to access and
perform unauthorized operations by accessing those URLs directly."
web2py maps URL requests to Python modules and functions. web2py provides
a mechanism for declaring which functions are public and which require
authentication and authorization. The included Role Based Access Control
API allow developers to restrict access to any function based on login, group
membership or group based permissions. The permissions are very granular and
can be combined with CRUD to allow, for example, to give access to specific tables
and/or records. web2py also allows digitally signed URL and provides API to
digitally sign ajax callbacks.
34 web2py full-stack web framework,4th edition
web2py was reviewed for security and you can find the result of the review
in ref. [20].
1.6In the box
You can download web2py from the official web site:
web2py is composed of the following components:
libraries: provide core functionality of web2py and are accessible
web server: the Rocket WSGI web server.
• the admin application: used to create, design, and manage other
web2py applications. admin provide a complete web-based Integrated
Development Environment (IDE) for building web2py applications. It also
includes other functionality, such as web-based testing and a web-based
• the examples application: contains documentation and interactive
examples. examples is a clone of the official web2py.com web site, and
includes epydoc documentation.
the welcome application: the basic scaffolding template for any other
application. By default it includes a pure CSS cascading menu and user
authentication (discussed in Chapter 9).
web2py is distributed in source code, and in binary form for Microsoft
Windows and for Mac OS X.
The source code distribution can be used in any platform where Python
runs and includes the above-mentioned components. To run the source
code, you need Python 2.5pre-installed on the system. You also need one
of the supported database engines installed. For testing and light-demand
applications, you can use the SQLite database, included with Python 2.5.
introduction 35
The binary versions of web2py (for Windows and Mac OS X) include a
Python 2.5interpreter and the SQLite database. Technically, these two are not
components of web2py. Including them in the binary distributions enables
you to run web2py out of the box.
The following image depicts the overall web2py structure:
web2py is licensed under the LGPL version 3License. The full text of the
license if available in ref. [31].
In accordance with LGPL you may:
redistribute web2py with your apps (including official web2py binary
release your applications which use official web2py libraries under any
license you wish
Yet you must:
make clear in the documentation that your application uses web2py
release any modification of the web2py libraries under the LGPLv3license
36 web2py full-stack web framework,4th edition
The license includes the usual disclaimer:
Earlier versions
Earlier versions of web2py, 1.0.*-1.90.*, were released under the GPL2license
plus a commercial exception which, for practical purposes, was very similar
to the current LPGLv3.
Third party software distributed with web2py web2py contains third party
software under the gluon/contrib/ folder and various JavaScript and CSS
files. These files are distributed with web2py under their original licenses, as
stated in the files.
introduction 37
web2py was originally developed by and copyrighted by Massimo Di Pierro.
The first version (1.0) was released in October, 2007. Since then it has been
adopted by many users, some of whom have also contributed bug reports,
testing, debugging, patches, and proofreading of this book.
Some of the major contributors are, in alphabetical order by first name:
Alexey Nezhdanov, Alvaro Justen, Andrew Willimott, Angelo Compagnucci,
Anthony Bastardi, Antonio Ramos, Arun K. Rajeevan, Attila Csipa, Bill
Ferret, Boris Manojlovic, Branko Vukelic, Brian Meredyk, Bruno Rocha,
Carlos Galindo, Carsten Haese, Chris Clark, Chris Steel, Christian Foster
Howes, Christopher Smiga, CJ Lazell, Cliff Kachinske, Craig Younkins,
Daniel Lin, David Harrison, David Wagner, Denes Lengyel, Douglas Soares
de Andrade, Eric Vicenti, Falko Krause, Farsheed Ashouri, Fran Boon,
Francisco Gama, Fred Yanowski, Gilson Filho, Graham Dumpleton, Gyuris
Szabolcs, Hamdy Abdel-Badeea, Hans Donner, Hans Murx, Hans C. v.
Stockhausen, Ian Reinhart Geiser, Ismael Serratos, Jan Beilicke, Jonathan
Benn, Jonathan Lundell, Josh Goldfoot, Jose Jachuf, Josh Jaques, José
Vicente de Sousa, Keith Yang, Kenji Hosoda, Kyle Smith, Limodou, Lucas
D’Ávila, Marcel Leuthi, Marcel Hellkamp, Marcello Della Longa, Mariano
Reingart, Mark Larsen, Mark Moore, Markus Gritsch, Martin Hufsky, Martin
Mulone, Mateusz Banach, Miguel Lopez, Michael Willis, Michele Comitini,
Nathan Freeze, Niall Sweeny, Niccolo Polo, Nicolas Bruxer, Olaf Ferger,
Omi Chiba, Ondrej Such, Ovidio Marinho Falcao Neto, Pai, Paolo Caruccio,
Patrick Breitenbach, Phyo Arkar Lwin, Pierre Thibault, Ramjee Ganti, Robin
Bhattacharyya, Ross Peoples, Ruijun Luo, Ryan Seto, Scott Roberts, Sergey
Podlesnyi, Sharriff Aina, Simone Bizzotto, Sriram Durbha, Sterling Hankins,
Stuart Rackham, Telman Yusupov, Thadeus Burgess, Tim Michelsen, Timothy
Farrell, Yair Eshel, Yarko Tymciurak, Younghyun Jo, Vidul Nikolaev Petrov,
Vinicius Assef, Zahariash.
I am sure I forgot somebody, so I apologize.
I particularly thank Jonathan, Mariano, Bruno, Martin, Nathan, Simone,
Thadeus, Tim, Iceberg, Denes, Hans, Christian, Fran and Patrick for
38 web2py full-stack web framework,4th edition
their major contributions to web2py and Anthony, Alvaro, Bruno, Denes,
Felipe, Graham, Jonathan, Hans, Kyle, Mark, Michele, Richard, Robin,
Roman, Scott, Shane, Sharriff, Sriram, Sterling, Stuart, Thadeus (and others)
for proofreading various versions of this book. Their contribution was
invaluable. If you find any errors in this book, they are exclusively my fault,
probably introduced by a last-minute edit. I also thank Ryan Steffen of Wiley
Custom Learning Solutions for help with publishing the first edition of this
book. web2py contains code from the following authors, whom I would like
to thank:
Guido van Rossum for Python [2], Peter Hunt, Richard Gordon, Timothy
Farrell for the Rocket [22] web server, Christopher Dolivet for EditArea [23],
Bob Ippolito for simplejson [25], Simon Cusack and Grant Edwards for
pyRTF [26], Dalke Scientific Software for pyRSS2Gen [27], Mark Pilgrim for
feedparser [28], Trent Mick for markdown2[29], Allan Saddi for fcgi.py, Evan
Martin for the Python memcache module [30], John Resig for jQuery [32].
The cover of this book was designed by Peter Kirchner at Young Designers.
I thank Helmut Epp (provost of DePaul University), David Miller (Dean of
the College of Computing and Digital Media of DePaul University), and
Estia Eichten (Member of MetaCryption LLC), for their continuous trust and
Finally, I wish to thank my wife, Claudia, and my son, Marco, for putting up
with me during the many hours I have spent developing web2py, exchanging
emails with users and collaborators, and writing this book. This book is
dedicated to them.
1.9About this book
This book includes the following chapters, besides this introduction:
Chapter 2is a minimalist introduction to Python. It assumes knowledge of
both procedural and object-oriented programming concepts such as loops,
conditions, function calls and classes, and covers basic Python syntax. It
also covers examples of Python modules that are used throughout the
introduction 39
book. If you already know Python, you may skip Chapter 2.
• Chapter 3shows how to start web2py, discusses the administrative
interface, and guides the reader through various examples of increasing
complexity: an application that returns a string, a counter application, an
image blog, and a full blown wiki application that allows image uploads
and comments, provides authentication, authorization, web services and
an RSS feed. While reading this chapter, you may need to refer to Chapter
2for general Python syntax and to the following chapters for a more
detailed reference about the functions that are used.
Chapter 4covers more systematically the core structure and libraries: URL
mapping, request, response, sessions, caching, cron, internationalization
and general workflow.
Chapter 5is a reference for the template language used to build views. It
shows how to embed Python code into HTML, and demonstrates the use
of helpers (objects that can generate HTML).
Chapter 6covers the Database Abstraction Layer, or DAL. The syntax of
the DAL is presented through a series of examples.
Chapter 7covers forms, form validation and form processing. FORM is
the low level helper for form building. SQLFORM is the high level form
builder. In Chapter 7we also discuss the Create/Read/Update/Delete
Chapter 8covers communication with as sending emails and SMSes.
Chapter 9covers authentication, authorization and the extensible Role-
Based Access Control mechanism available in web2py. Mail configuration
and CAPTCHA are also discussed here, since they are used for
authentication. In the third edition of the book we have added extensive
coverage of integration with third-party authentication mechanisms such
as OpenID, OAuth, Google, Facebook, LinkedIn, etc.
Chapter 10 is about creating web services in web2py. We provide examples
of integration with the Google Web Toolkit via Pyjamas, and with Adobe
Flash via PyAMF.
40 web2py full-stack web framework,4th edition
Chapter 11 is about web2py and jQuery recipes. web2py is designed
mainly for server-side programming, but it includes jQuery, since we have
found it to be the best open-source JavaScript library available for effects
and Ajax. In this chapter, we discuss how to effectively use jQuery with
Chapter 12 discusses web2py components and plugins as a way to build
modular applications. We provide an example of a plugin that implements
many commonly used functionality, such as charting, comments, tagging,
and wiki.
Chapter 13 is about production deployment of web2py applications. We
mainly address three possible production scenarios: on a Linux web
server or a set of servers (which we consider the main deployment
alternative), running as a service on a Microsoft Windows environment,
and deployment on the Google Applications Engine. In this chapter, we
also discuss security and scalability issues.
Chapter 14 contains a variety of other recipes to solve specific tasks,
including upgrades, geocoding, pagination, the Twitter API, and more.
This book only covers basic web2py functionalities and the API that ships
with web2py. This book does not cover web2py appliances (i.e. ready made
You can download web2py appliances from the corresponding web site [34].
You can find additional topics discussed on AlterEgo [35], the interactive
web2py FAQ.
This book has been written using the markmin syntax and automatically
converted to HTML, LaTeX and PDF.
1.10 Elements of style
PEP8[36] contains good style practices when programming with Python.
You will find that web2py does not always follow these rules. This is not
because of omissions or negligence; it is our belief that the users of web2py
introduction 41
should follow these rules and we encourage it. We chose not to follow some
of those rules when defining web2py helper objects in order to minimize the
probability of name conflict with objects defined by the user.
For example, the class that represents a <div> is called DIV, while according
to the Python style reference it should have been called Div. We believe that,
for this specific example that using an all-upper-case "DIV" is a more natural
choice. Moreover, this approach leaves programmers free to create a class
called "Div" if they choose to do so. Our syntax also maps naturally into the
DOM notation of most browsers (including, for example, Firefox).
According to the Python style guide, all-upper-case strings should be
used for constants and not variables. Continuing with our example, even
considering that DIV is a class, it is a special class that should never
be modified by the user because doing so would break other web2py
applications. Hence, we believe this qualifies the DIV class as something that
should be treated as a constant, further justifying our choice of notation.
In summary, the following conventions are followed:
HTML helpers and validators are all upper case for the reasons discussed
above (for example DIV,A,FORM,URL).
The translator object Tis upper case despite the fact that it is an instance
of a class and not a class itself. Logically the translator object performs
an action similar to the HTML helpers, it affects rendering part of the
presentation. Also, Tneeds to be easy to locate in the code and must have
a short name.
DAL classes follow the Python style guide (first letter capitalized), for
example Table,Field,Query,Row,Rows, etc.
In all other cases we believe we have followed, as much as possible, the
Python Style Guide (PEP8). For example all instance objects are lower-case
(request, response, session, cache), and all internal classes are capitalized.
In all the examples of this book, web2py keywords are shown in bold, while
strings and comments are shown in italic.
The Python language
2.1About Python
Python is a general-purpose high-level programming language. Its design
philosophy emphasizes programmer productivity and code readability. It
has a minimalist core syntax with very few basic commands and simple
semantics, but it also has a large and comprehensive standard library,
including an Application Programming Interface (API) to many of the
underlying operating system (OS) functions. Python code, while minimalist,
defines built-in objects such as linked lists (list), tuples (tuple), hash tables
(dict), and arbitrarily long integers (long).
Python supports multiple programming paradigms, including object-
oriented (class), imperative (def), and functional (lambda) programming.
Python has a dynamic type system and automatic memory management
using reference counting (similar to Perl, Ruby, and Scheme).
Python was first released by Guido van Rossum in 1991. The language has
an open, community-based development model managed by the non-profit
Python Software Foundation. There are many interpreters and compilers that
implement the Python language, including one in Java (Jython) but, in this
brief review, we refer to the reference C implementation created by Guido.
You can find many tutorials, the official documentation and library references
44 web2py full-stack web framework,4th edition
of the language on the official Python website. [2]
For additional Python references, we can recommend the books in ref. [37]
and ref. [38].
You may skip this chapter if you are already familiar with the Python
2.2Starting up
The binary distributions of web2py for Microsoft Windows or Apple OS X
come packaged with the Python interpreter built into the distribution file
You can start it on Windows with the following command (type at the DOS
1web2py.exe -S welcome
On Apple OS X, enter the following command type in a Terminal window
(assuming you’re in the same folder as web2py.app):
1./web2py.app/Contents/MacOS/web2py -S welcome
On a Linux or other Unix box, chances are that you have Python already
installed. If so, at a shell prompt type:
1python web2py.py -S welcome
If you do not have Python 2.5(or later 2.x) already installed, you will have to
download and install it before running web2py.
The -S welcome command line option instructs web2py to run the interactive
shell as if the commands were executed in a controller for the welcome
application, the web2py scaffolding application. This exposes almost all
web2py classes, objects and functions to you. This is the only difference
between the web2py interactive command line and the normal Python
command line.
The admin interface also provides a web-based shell for each application.
You can access the one for the "welcome" application at.
the python language 45
You can try all the examples in this chapter using the normal shell or the
web-based shell.
2.3help, dir
The Python language provides two commands to obtain documentation
about objects defined in the current scope, both built-in and user-defined.
We can ask for help about an object, for example "1":
1>>> help(1)
2Help on int object:
4class int(object)
5| int(x[, base]) -> integer
7| Convert a string or number to an integer, if possible. Afloating point
8| argument will be truncated towards zero (this does not include a string
9| representation of a floating point number!) When converting a string, use
10 | the optional base. It is an error to supply a base when converting a
11 | non-string. If the argument is outside the integer range a long object
12 | will be returned instead.
13 |
14 | Methods defined here:
15 |
16 |__abs__(...)
17 | x.__abs__() <==> abs(x)
18 ...
and, since "1" is an integer, we get a description about the int class and all
its methods. Here the output has been truncated because it is very long and
Similarly, we can obtain a list of methods of the object "1" with the command
1>>> dir(1)
46 web2py full-stack web framework,4th edition
10 '__rshift__','__rsub__','__rtruediv__','__rxor__','__setattr__',
11 '__str__','__sub__','__truediv__','__xor__']
Python is a dynamically typed language, meaning that variables do not have
a type and therefore do not have to be declared. Values, on the other hand,
do have a type. You can query a variable for the type of value it contains:
1>>> a = 3
2>>> print type(a)
3<type 'int'>
4>>> a = 3.14
5>>> print type(a)
6<type 'float'>
7>>> a = 'hello python'
8>>> print type(a)
9<type 'str'>
Python also includes, natively, data structures such as lists and dictionaries.
Python supports the use of two different types of strings: ASCII strings
and Unicode strings. ASCII strings are delimited by ’...’, "..." or by ’..’ or
"""...""". Triple quotes delimit multiline strings. Unicode strings start with a u
followed by the string containing Unicode characters. A Unicode string can
be converted into an ASCII string by choosing an encoding for example:
1>>> a = 'this is an ASCII string'
2>>> b = u'This is a Unicode string'
3>>> a = b.encode('utf8')
After executing these three commands, the resulting ais an ASCII string
storing UTF8encoded characters. By design, web2py uses UTF8encoded
strings internally.
the python language 47
It is also possible to write variables into strings in various ways:
1>>> print 'number is ' + str(3)
2number is 3
3>>> print 'number is %s' % (3)
4number is 3
5>>> print 'number is %(number)s' % dict(number=3)
6number is 3
The last notation is more explicit and less error prone, and is to be preferred.
Many Python objects, for example numbers, can be serialized into strings
using str or repr. These two commands are very similar but produce slightly
different output. For example:
1>>> for i in [3, 'hello']:
2print str(i), repr(i)
33 3
4hello 'hello'
For user-defined classes, str and repr can be defined/redefined using the
special operators __str__ and __repr__. These are briefly described later on;
for more, refer to the official Python documentation [39]. repr always has a
default value.
Another important characteristic of a Python string is that, like a list, it is an
iterable object.
1>>> for i in 'hello':
2print i
The main methods of a Python list are append, insert, and delete:
1>>> a = [1, 2, 3]
2>>> print type(a)
3<type 'list'>
4>>> a.append(8)
48 web2py full-stack web framework,4th edition
5>>> a.insert(2, 7)
6>>> del a[0]
7>>> print a
8[2, 7, 3, 8]
9>>> print len(a)
10 4
Lists can be sliced:
1>>> print a[:3]
2[2, 7, 3]
3>>> print a[1:]
4[7, 3, 8]
5>>> print a[-2:]
6[3, 8]
and concatenated:
1>>> a = [2, 3]
2>>> b = [5, 6]
3>>> print a + b
4[2, 3, 5, 6]
A list is iterable; you can loop over it:
1>>> a = [1, 2, 3]
2>>> for i in a:
3print i
The elements of a list do not have to be of the same type; they can be any
type of Python object.
There is a very common situation for which a list comprehension can be used.
Consider the following code:
1>>> a = [1,2,3,4,5]
2>>> b = []
3>>> for x in a:
4if x % 2 == 0:
5b.append(x *3)
6>>> b
7[6, 12]
the python language 49
This code clearly processes a list of items, selects and modifies a subset of the
input list, and creates a new result list, and this code can be entirely replaced
with the following list comprehension:
1>>> a = [1,2,3,4,5]
2>>> b = [x *3 for x in a if x % 2 == 0]
3>>> b
4[6, 12]
A tuple is like a list, but its size and elements are immutable, while in a list
they are mutable. If a tuple element is an object, the object attributes are
mutable. A tuple is delimited by round brackets.
1>>> a = (1, 2, 3)
So while this works for a list:
1>>> a = [1, 2, 3]
2>>> a[1] = 5
3>>> print a
4[1, 5, 3]
the element assignment does not work for a tuple:
1>>> a = (1, 2, 3)
2>>> print a[1]
4>>> a[1] = 5
5Traceback (most recent call last):
6File "<stdin>", line 1, in <module>
7TypeError: 'tuple' object does not support item assignment
A tuple, like a list, is an iterable object. Notice that a tuple consisting of a
single element must include a trailing comma, as shown below:
1>>> a = (1)
2>>> print type(a)
3<type 'int'>
4>>> a = (1,)
5>>> print type(a)
6<type 'tuple'>
50 web2py full-stack web framework,4th edition
Tuples are very useful for efficient packing of objects because of their
immutability, and the brackets are often optional:
1>>> a = 2, 3, 'hello'
2>>> x, y, z = a
3>>> print x
5>>> print z
A Python dict-ionary is a hash table that maps a key object to a value object.
For example:
1>>> a = {'k':'v','k2':3}
2>>> a['k']
4>>> a['k2']
6>>> a.has_key('k')
8>>> a.has_key('v')
Keys can be of any hashable type (int, string, or any object whose class
implements the __hash__ method). Values can be of any type. Different keys
and values in the same dictionary do not have to be of the same type. If the
keys are alphanumeric characters, a dictionary can also be declared with the
alternative syntax:
1>>> a = dict(k='v', h2=3)
2>>> a['k']
4>>> print a
Useful methods are has_key,keys,values and items:
1>>> a = dict(k='v', k2=3)
2>>> print a.keys()
4>>> print a.values()
5['v', 3]
the python language 51
6>>> print a.items()
7[('k','v'), ('k2', 3)]
The items method produces a list of tuples, each containing a key and its
associated value.
Dictionary elements and list elements can be deleted with the command del:
1>>> a = [1, 2, 3]
2>>> del a[1]
3>>> print a
4[1, 3]
5>>> a = dict(k='v', h2=3)
6>>> del a['h2']
7>>> print a
Internally, Python uses the hash operator to convert objects into integers, and
uses that integer to determine where to store the value.
1>>> hash("hello world")
2.5About indentation
Python uses indentation to delimit blocks of code. A block starts with a
line ending in colon, and continues for all lines that have a similar or higher
indentation as the next line. For example:
1>>> i = 0
2>>> while i < 3:
3>>> print i
4>>> i = i + 1
It is common to use four spaces for each level of indentation. It is a good
policy not to mix tabs with spaces, which can result in (invisible) confusion.
52 web2py full-stack web framework,4th edition
In Python, you can loop over iterable objects:
1>>> a = [0, 1, 'hello','python']
2>>> for i in a:
3print i
One common shortcut is xrange, which generates an iterable range without
storing the entire list of elements.
1>>> for i in xrange(0, 4):
2print i
This is equivalent to the C/C++/C#/Java syntax:
1for(int i=0; i<4; i=i+1) { print(i); }
Another useful command is enumerate, which counts while looping:
1>>> a = [0, 1, 'hello','python']
2>>> for i, j in enumerate(a):
3print i, j
40 0
51 1
62 hello
73 python
There is also a keyword range(a, b, c) that returns a list of integers starting
with the value a, incrementing by c, and ending with the last value smaller
than b,adefaults to 0and cdefaults to 1.xrange is similar but does not
actually generate the list, only an iterator over the list; thus it is better for
You can jump out of a loop using break
1>>> for i in [1, 2, 3]:
2print i
the python language 53
You can jump to the next loop iteration without executing the entire code
block with continue
1>>> for i in [1, 2, 3]:
2print i
4print 'test'
The while loop in Python works much as it does in many other programming
languages, by looping an indefinite number of times and testing a condition
before each iteration. If the condition is False, the loop ends.
1>>> i = 0
2>>> while i < 10:
4>>> print i
There is no loop...until construct in Python.
The use of conditionals in Python is intuitive:
1>>> for i in range(3):
2>>> if i == 0:
3>>> print 'zero'
4>>> elif i == 1:
5>>> print 'one'
6>>> else:
7>>> print 'other'
10 other
54 web2py full-stack web framework,4th edition
"elif" means "else if". Both elif and else clauses are optional. There can be
more than one elif but only one else statement. Complex conditions can be
created using the not,and and or operators.
1>>> for i in range(3):
2>>> if i == 0 or (i == 1 and i + 1 == 2):
3>>> print '0 or 1'
Python can throw - pardon, raise - Exceptions:
1>>> try:
2>>> a = 1 / 0
3>>> except Exception, e:
4>>> print 'oops: %s' % e
5>>> else:
6>>> print 'no problem here'
7>>> finally:
8>>> print 'done'
9oops: integer division or modulo by zero
10 done
If the exception is raised, it is caught by the except clause, which is executed,
while the else clause is not. If no exception is raised, the except clause is not
executed, but the else one is. The finally clause is always executed.
There can be multiple except clauses for different possible exceptions:
1>>> try:
2>>> raise SyntaxError
3>>> except ValueError:
4>>> print 'value error'
5>>> except SyntaxError:
6>>> print 'syntax error'
7syntax error
The else and finally clauses are optional.
Here is a list of built-in Python exceptions + HTTP (defined by web2py)
2+-- HTTP (defined by web2py)
3+-- SystemExit
the python language 55
4+-- KeyboardInterrupt
5+-- Exception
6+-- GeneratorExit
7+-- StopIteration
8+-- StandardError
9| +-- ArithmeticError
10 | | +-- FloatingPointError
11 | | +-- OverflowError
12 | | +-- ZeroDivisionError
13 | +-- AssertionError
14 | +-- AttributeError
15 | +-- EnvironmentError
16 | | +-- IOError
17 | | +-- OSError
18 | | +-- WindowsError (Windows)
19 | | +-- VMSError (VMS)
20 | +-- EOFError
21 | +-- ImportError
22 | +-- LookupError
23 | | +-- IndexError
24 | | +-- KeyError
25 | +-- MemoryError
26 | +-- NameError
27 | | +-- UnboundLocalError
28 | +-- ReferenceError
29 | +-- RuntimeError
30 | | +-- NotImplementedError
31 | +-- SyntaxError
32 | | +-- IndentationError
33 | | +-- TabError
34 | +-- SystemError
35 | +-- TypeError
36 | +-- ValueError
37 | | +-- UnicodeError
38 | | +-- UnicodeDecodeError
39 | | +-- UnicodeEncodeError
40 | | +-- UnicodeTranslateError
41 +-- Warning
42 +-- DeprecationWarning
43 +-- PendingDeprecationWarning
44 +-- RuntimeWarning
45 +-- SyntaxWarning
46 +-- UserWarning
47 +-- FutureWarning
48 +-- ImportWarning
49 +-- UnicodeWarning
56 web2py full-stack web framework,4th edition
For a detailed description of each of them, refer to the official Python
documentation. web2py exposes only one new exception, called HTTP. When
raised, it causes the program to return an HTTP error page (for more on this
refer to Chapter 4).
Any object can be raised as an exception, but it is good practice to raise
objects that extend one of the built-in exception classes.
2.10 def...return
Functions are declared using def. Here is a typical Python function:
1>>> def f(a, b):
2return a + b
3>>> print f(4, 2)
There is no need (or way) to specify types of the arguments or the return
type(s). In this example, a function fis defined that can take two arguments.
Functions are the first code syntax feature described in this chapter to
introduce the concept of scope, or namespace. In the above example, the
identifiers aand bare undefined outside of the scope of function f:
1>>> def f(a):
2return a + 1
3>>> print f(1)
5>>> print a
6Traceback (most recent call last):
7File "<pyshell#22>", line 1, in <module>
8print a
9NameError: name 'a' is not defined
Identifiers defined outside of function scope are accessible within the
function; observe how the identifier ais handled in the following code:
1>>> a = 1
2>>> def f(b):
3return a + b
4>>> print f(1)
6>>> a = 2
the python language 57
7>>> print f(1) # new value of a is used
9>>> a = 1 # reset a
10 >>> def g(b):
11 a=2# creates a new local a
12 return a + b
13 >>> print g(2)
14 4
15 >>> print a # global a is unchanged
16 1
If ais modified, subsequent function calls will use the new value of the global
abecause the function definition binds the storage location of the identifier
a, not the value of aitself at the time of function declaration; however, if a
is assigned-to inside function g, the global ais unaffected because the new
local ahides the global value. The external-scope reference can be used in
the creation of closures:
1>>> def f(x):
2def g(y):
3return x *y
4return g
5>>> doubler = f(2) # doubler is a new function
6>>> tripler = f(3) # tripler is a new function
7>>> quadrupler = f(4) # quadrupler is a new function
8>>> print doubler(5)
10 >>> print tripler(5)
11 15
12 >>> print quadrupler(5)
13 20
Function fcreates new functions; and note that the scope of the name gis
entirely internal to f. Closures are extremely powerful.
Function arguments can have default values, and can return multiple results:
1>>> def f(a, b=2):
2return a + b, a - b
3>>> x, y = f(5)
4>>> print x
6>>> print y
58 web2py full-stack web framework,4th edition
Function arguments can be passed explicitly by name, and this means that
the order of arguments specified in the caller can be different than the order
of arguments with which the function was defined:
1>>> def f(a, b=2):
2return a + b, a - b
3>>> x, y = f(b=5, a=2)
4>>> print x
6>>> print y
Functions can also take a runtime-variable number of arguments:
1>>> def f(*a, **b):
2return a, b
3>>> x, y = f(3, 'hello', c=4, test='world')
4>>> print x
5(3, 'hello')
6>>> print y
7{'c':4, 'test':'world'}
Here arguments not passed by name (3, ’hello’) are stored in the tuple a, and
arguments passed by name (cand test) are stored in the dictionary b.
In the opposite case, a list or tuple can be passed to a function that requires
individual positional arguments by unpacking them:
1>>> def f(a, b):
2return a + b
3>>> c = (1, 2)
4>>> print f(*c)
and a dictionary can be unpacked to deliver keyword arguments:
1>>> def f(a, b):
2return a + b
3>>> c = {'a':1, 'b':2}
4>>> print f(**c)
lambda provides a way to create a very short unnamed function very easily:
the python language 59
1>>> a = lambda b: b + 2
2>>> print a(3)
The expression "lambda [a]:[b]" literally reads as "a function with arguments
[a] that returns [b]". The lambda expression is itself unnamed, but the function
acquires a name by being assigned to identifier a. The scoping rules for def
apply to lambda equally, and in fact the code above, with respect to a, is
identical to the function declaration using def:
1>>> def a(b):
2return b + 2
3>>> print a(3)
The only benefit of lambda is brevity; however, brevity can be very convenient
in certain situations. Consider a function called map that applies a function to
all items in a list, creating a new list:
1>>> a = [1, 7, 2, 5, 4, 8]
2>>> map(lambda x: x + 2, a)
3[3, 9, 4, 7, 6, 10]
This code would have doubled in size had def been used instead of lambda.
The main drawback of lambda is that (in the Python implementation) the
syntax allows only for a single expression; however, for longer functions,
def can be used and the extra cost of providing a function name decreases as
the length of the function grows. Just like def,lambda can be used to curry
functions: new functions can be created by wrapping existing functions such
that the new function carries a different set of arguments:
1>>> def f(a, b): return a + b
2>>> g = lambda a: f(a, 3)
3>>> g(2)
There are many situations where currying is useful, but one of those is
directly useful in web2py: caching. Suppose you have an expensive function
that checks whether its argument is prime:
1def isprime(number):
2for p in range(2, number):
3if (number % p) == 0:
60 web2py full-stack web framework,4th edition
4return False
5return True
This function is obviously time consuming.
Suppose you have a caching function cache.ram that takes three arguments: a
key, a function and a number of seconds.
1value = cache.ram('key', f, 60)
The first time it is called, it calls the function f(), stores the output in a
dictionary in memory (let’s say "d"), and returns it so that value is:
1value = d['key']=f()
The second time it is called, if the key is in the dictionary and not older
than the number of seconds specified (60), it returns the corresponding value
without performing the function call.
1value = d['key']
How would you cache the output of the function isprime for any input? Here
is how:
1>>> number = 7
2>>> seconds = 60
3>>> print cache.ram(str(number), lambda: isprime(number), seconds)
5>>> print cache.ram(str(number), lambda: isprime(number), seconds)
The output is always the same, but the first time cache.ram is called, isprime
is called; the second time it is not.
Python functions, created with either def or lambda allow re-factoring existing
functions in terms of a different set of arguments. cache.ram and cache.disk
are web2py caching functions.
2.11 class
Because Python is dynamically typed, Python classes and objects may seem
odd. In fact, you do not need to define the member variables (attributes)
the python language 61
when declaring a class, and different instances of the same class can have
different attributes. Attributes are generally associated with the instance, not
the class (except when declared as "class attributes", which is the same as
"static member variables" in C++/Java).
Here is an example:
1>>> class MyClass(object): pass
2>>> myinstance = MyClass()
3>>> myinstance.myvariable = 3
4>>> print myinstance.myvariable
Notice that pass is a do-nothing command. In this case it is used to define a
class MyClass that contains nothing. MyClass() calls the constructor of the class
(in this case the default constructor) and returns an object, an instance of the
class. The (object) in the class definition indicates that our class extends the
built-in object class. This is not required, but it is good practice.
Here is a more complex class:
1>>> class MyClass(object):
2>>> z = 2
3>>> def __init__(self, a, b):
4>>> self.x = a, self.y = b
5>>> def add(self):
6>>> return self.x + self.y + self.z
7>>> myinstance = MyClass(3, 4)
8>>> print myinstance.add()
Functions declared inside the class are methods. Some methods have special
reserved names. For example, __init__ is the constructor. All variables are
local variables of the method except variables declared outside methods. For
example, zis a class variable, equivalent to a C++ static member variable that
holds the same value for all instances of the class.
Notice that __init__ takes 3arguments and add takes one, and yet we call
them with 2and 0arguments respectively. The first argument represents,
by convention, the local name used inside the method to refer to the current
object. Here we use self to refer to the current object, but we could have used
any other name. self plays the same role as *this in C++ or this in Java, but
self is not a reserved keyword.
62 web2py full-stack web framework,4th edition
This syntax is necessary to avoid ambiguity when declaring nested classes,
such as a class that is local to a method inside another class.
2.12 Special attributes, methods and operators
Class attributes, methods, and operators starting with a double underscore
are usually intended to be private (i.e. to be used internally but not exposed
outside the class) although this is a convention that is not enforced by the
Some of them are reserved keywords and have a special meaning.
Here, as an example, are three of them:
They can be used, for example, to create a container object that acts like a list:
1>>> class MyList(object):
2>>> def __init__(self, *a): self.a = list(a)
3>>> def __len__(self): return len(self.a)
4>>> def __getitem__(self, i): return self.a[i]
5>>> def __setitem__(self, i, j): self.a[i] = j
6>>> b = MyList(3, 4, 5)
7>>> print b[1]
9>>> b.a[1] = 7
10 >>> print b.a
11 [3, 7, 5]
Other special operators include __getattr__ and __setattr__, which define the
get and set attributes for the class, and __sum__ and __sub__, which overload
arithmetic operators. For the use of these operators we refer the reader to
more advanced books on this topic. We have already mentioned the special
operators __str__ and __repr__.
the python language 63
2.13 File input/output
In Python you can open and write in a file with:
1>>> file = open('myfile.txt','w')
2>>> file.write('hello world')
3>>> file.close()
Similarly, you can read back from the file with:
1>>> file = open('myfile.txt','r')
2>>> print file.read()
3hello world
Alternatively, you can read in binary mode with "rb", write in binary mode
with "wb", and open the file in append mode "a", using standard C notation.
The read command takes an optional argument, which is the number of bytes.
You can also jump to any location in a file using seek.
You can read back from the file with read
1>>> print file.seek(6)
2>>> print file.read()
and you can close the file with:
1>>> file.close()
In the standard distribution of Python, which is known as CPython, variables
are reference-counted, including those holding file handles, so CPython knows
that when the reference count of an open file handle decreases to zero, the file
may be closed and the variable disposed. However, in other implementations of
Python such as PyPy, garbage collection is used instead of reference counting,
and this means that it is possible that there may accumulate too many open file
handles at one time, resulting in an error before the gc has a chance to close
and dispose of them all. Therefore it is best to explicitly close file handles when
they are no longer needed. web2py provides two helper functions, read_file()
and write_file() inside the gluon.fileutils namespace that encapsulate the
file access and ensure that the file handles being used are properly closed.
64 web2py full-stack web framework,4th edition
When using web2py, you do not know where the current directory is, because
it depends on how web2py is configured. The variable request.folder contains
the path to the current application. Paths can be concatenated with the
command os.path.join, discussed below.
2.14 exec,eval
Unlike Java, Python is a truly interpreted language. This means it has the
ability to execute Python statements stored in strings. For example:
1>>> a = "print 'hello world'"
2>>> exec(a)
3'hello world'
What just happened? The function exec tells the interpreter to call itself and
execute the content of the string passed as argument. It is also possible to
execute the content of a string within a context defined by the symbols in a
1>>> a = "print b"
2>>> c = dict(b=3)
3>>> exec(a, {}, c)
Here the interpreter, when executing the string a, sees the symbols defined
in c(bin the example), but does not see cor athemselves. This is different
than a restricted environment, since exec does not limit what the inner code
can do; it just defines the set of variables visible to the code.
A related function is eval, which works very much like exec except that it
expects the argument to evaluate to a value, and it returns that value.
1>>> a = "3*4"
2>>> b = eval(a)
3>>> print b
the python language 65
2.15 import
The real power of Python is in its library modules. They provide a large and
consistent set of Application Programming Interfaces (APIs) to many system
libraries (often in a way independent of the operating system).
For example, if you need to use a random number generator, you can do:
1>>> import random
2>>> print random.randint(0, 9)
This prints a random integer between 0and 9(including 9), 5in the example.
The function randint is defined in the module random. It is also possible to
import an object from a module into the current namespace:
1>>> from random import randint
2>>> print randint(0, 9)
or import all objects from a module into the current namespace:
1>>> from random import *
2>>> print randint(0, 9)
or import everything in a newly defined namespace:
1>>> import random as myrand
2>>> print myrand.randint(0, 9)
In the rest of this book, we will mainly use objects defined in modules os,
sys,datetime,time and cPickle.
All of the web2py objects are accessible via a module called gluon, and that is
the subject of later chapters. Internally, web2py uses many Python modules
(for example thread), but you rarely need to access them directly.
In the following subsections we consider those modules that are most useful.
This module provides an interface to the operating system API. For example:
66 web2py full-stack web framework,4th edition
1>>> import os
2>>> os.chdir('..')
3>>> os.unlink('filename_to_be_deleted')
Some of the os functions, such as chdir, MUST NOT be used in web2py
because they are not thread-safe.
os.path.join is very useful; it allows the concatenation of paths in an OS-
independent way:
1>>> import os
2>>> a = os.path.join('path','sub_path')
3>>> print a
System environment variables can be accessed via:
1>>> print os.environ
which is a read-only dictionary.
The sys module contains many variables and functions, but the one we use
the most is sys.path. It contains a list of paths where Python searches for
modules. When we try to import a module, Python looks for it in all the
folders listed in sys.path. If you install additional modules in some location
and want Python to find them, you need to append the path to that location
to sys.path.
1>>> import sys
2>>> sys.path.append('path/to/my/modules')
When running web2py, Python stays resident in memory, and there is only
one sys.path, while there are many threads servicing the HTTP requests. To
avoid a memory leak, it is best to check if a path is already present before
1>>> path = 'path/to/my/modules'
2>>> if not path in sys.path:
the python language 67
The use of the datetime module is best illustrated by some examples:
1>>> import datetime
2>>> print datetime.datetime.today()
32008-07-04 14:03:90
4>>> print datetime.date.today()
Occasionally you may need to time-stamp data based on the UTC time as
opposed to local time. In this case you can use the following function:
1>>> import datetime
2>>> print datetime.datetime.utcnow()
32008-07-04 14:03:90
The datetime module contains various classes: date, datetime, time and
timedelta. The difference between two date or two datetime or two time
objects is a timedelta:
1>>> a = datetime.datetime(2008, 1, 1, 20, 30)
2>>> b = datetime.datetime(2008, 1, 2, 20, 30)
3>>> c = b - a
4>>> print c.days
In web2py, date and datetime are used to store the corresponding SQL types
when passed to or returned from the database.
The time module differs from date and datetime because it represents time as
seconds from the epoch (beginning of 1970).
1>>> import time
2>>> t = time.time()
Refer to the Python documentation for conversion functions between time in
seconds and time as a datetime.
68 web2py full-stack web framework,4th edition
This is a very powerful module. It provides functions that can serialize
almost any Python object, including self-referential objects. For example,
let’s build a weird object:
1>>> class MyClass(object): pass
2>>> myinstance = MyClass()
3>>> myinstance.x = 'something'
4>>> a = [1 ,2, {'hello':'world'}, [3, 4, [myinstance]]]
and now:
1>>> import cPickle
2>>> b = cPickle.dumps(a)
3>>> c = cPickle.loads(b)
In this example, bis a string representation of a, and cis a copy of agenerated
by de-serializing b. cPickle can also serialize to and de-serialize from a file:
1>>> cPickle.dump(a, open('myfile.pickle','wb'))
2>>> c = cPickle.load(open('myfile.pickle','rb'))
web2py comes in binary packages for Windows and Mac OS X. They include
the Python interpreter so you do not need to have it pre-installed. There
is also a source code version that runs on Windows, Mac, Linux, and other
Unix systems. The Windows and OS X binary versions include the necessary
Python interpreter. The source code package assumes that Python is already
installed on the computer. web2py requires no installation. To get started,
unzip the downloaded zip file for your specific operating system and execute
the corresponding web2py file.
On Windows, run:
On OS X, run:
1open web2py.app
On Unix and Linux, run from source by typing:
1python2.5 web2py.py
To run web2py on Windows from source install first Mark Hammond’s "Python for
Windows extensions, then run:
70 web2py full-stack web framework,4th edition
1python2.5 web2py.py
The web2py program accepts various command line options which are
discussed later.
By default, at startup, web2py displays a startup window and then displays
a GUI widget that asks you to choose a one-time administrator password,
the IP address of the network interface to be used for the web server, and a
port number from which to serve requests. By default, web2py runs its web
server on (port 8000 on localhost), but you can run it on any
available IP address and port. You can query the IP address of your network
interface by opening a command line and typing ipconfig on Windows or
ifconfig on OS X and Linux. From now on we assume web2py is running
on localhost ( Use to run web2py publicly on any of
your network interfaces.
If you do not provide an administrator password, the administration interface
is disabled. This is a security measure to prevent publicly exposing the admin
The administrative interface, admin, is only accessible from localhost unless
you run web2py behind Apache with mod_proxy. If admin detects a proxy,
the session cookie is set to secure and admin login does not work unless the
communication between the client and the proxy goes over HTTPS; this is a
security measure. All communications between the client and admin must
always be local or encrypted; otherwise an attacker would be able to perform
a man-in-the middle attack or a replay attack and execute arbitrary code on
overview 71
the server.
After the administration password has been set, web2py starts up the web
browser at the page:
If the computer does not have a default browser, open a web browser and
enter the URL.
Clicking on "administrative interface" takes you to the login page for the
administration interface.
The administrator password is the password you chose at startup. Notice
that there is only one administrator, and therefore only one administrator
password. For security reasons, the developer is asked to choose a new
password every time web2py starts unless the <recycle> option is specified.
72 web2py full-stack web framework,4th edition
This is distinct from the authentication mechanism in web2py applications.
After the administrator logs into web2py, the browser is redirected to the
"site" page.
This page lists all installed web2py applications and allows the administrator
to manage them. web2py comes with three applications:
An admin application, the one you are using right now.
An examples application, with the online interactive documentation and
a replica of the web2py official website.
A welcome application. This is the basic template for any other web2py
application. It is referred to as the scaffolding application. This is also the
application that welcomes a user at startup.
Ready-to-use web2py applications are referred to as web2py appliances. You
can download many freely available appliances from [34]. web2py users are
encouraged to submit new appliances, either in open-source or closed-source
(compiled and packed) form.
From the admin application’s site page, you can perform the following
install an application by completing the form on the bottom right of the
overview 73
page. Give a name to the application, select the file containing a packaged
application or the URL where the application is located, and click "submit".
uninstall an application by clicking the corresponding button. There is a
confirmation page.
create a new application by choosing a name and clicking "create".
package an application for distribution by clicking on the corresponding
button. A downloaded application is a tar file containing everything,
including the database. You should not untar this file; it is automatically
unpackaged by web2py when installed with admin.
clean up an application’s temporary files, such as sessions, errors and
cache files.
EDIT an application.
When you create a new application using admin, it starts as a clone of
the "welcome" scaffolding app with a "models/db.py" that creates a SQLite
database, connects to it, instantiates Auth, Crud and Service, configures
them. It also provides a "controller/default.py" which exposes actions "index",
"download", "user" for user management, and "call" for services. In the
following, we assume that these files have been removed; we will be creating
apps from scratch.
web2py also comes with a wizard, described later in this chapter, that can write
an alternate scaffolding code for you based on layouts and plugins available on
the web and based on high level description of the models.
3.2Say hello
Here, as an example, we create a simple web app that displays the message
"Hello from MyApp" to the user. We will call this application "myapp". We
will also add a counter that counts how many times the same user visits the
You can create a new application simply by typing its name in the form on
the top right of the site page in admin.
74 web2py full-stack web framework,4th edition
After you press [create], the application is created as a copy of the built-in
welcome application.
To run the new application, visit:
Now you have a copy of the welcome application.
To edit an application, click on the design button for the newly created
The Edit page tells you what is inside the application. Every web2py
application consists of certain files, most of which fall into one of isx
models: describe the data representation.
controllers: describe the application logic and workflow.
overview 75
views: describe the data presentation.
languages: describe how to translate the application presentation to other
modules: Python modules that belong to the application.
static files: static images, CSS files [40,41,42], JavaScript files [43,44], etc.
plugins: groups of files designed to work together.
Everything is neatly organized following the Model-View-Controller design
pattern. Each section in the edit page corresponds to a subfolder in the
application folder.
Notice that section headings will toggle their content. Folder names under
static files are also collapsible.
Each file listed in the section corresponds to a file physically located in the
subfolder. Any operation performed on a file via the admin interface (create,
edit, delete) can be performed directly from the shell using your favorite editor.
The application contains other types of files (database, session files, error
files, etc.), but they are not listed on the edit page because they are not
created or modified by the administrator; they are created and modified by
the application itself.
The controllers contain the logic and workflow of the application. Every
URL gets mapped into a call to one of the functions in the controllers
(actions). There are two default controllers: "appadmin.py" and "default.py".
appadmin provides the database administrative interface; we do not need
it now. "default.py" is the controller that you need to edit, the one that is
called by default when no controller is specified in the URL. Edit the "index"
function as follows:
1def index():
2return "Hello from MyApp"
Here is what the online editor looks like:
76 web2py full-stack web framework,4th edition
Save it and go back to the edit page. Click on the index link to visit the newly
created page.
When you visit the URL
the index action in the default controller of the myapp application is called.
It returns a string that the browser displays for us. It should look like this:
Now, edit the "index" function as follows:
1def index():
2return dict(message="Hello from MyApp")
Also from the edit page, edit the view "default/index.html" (the view file
associated with the action) and completely replace the existing contents of
that file with the following:
overview 77
Now the action returns a dictionary defining a message. When an action
returns a dictionary, web2py looks for a view with the name
[controller]/[function].[extension] and executes it. Here [extension] is the
requested extension. If no extension is specified, it defaults to "html", and
that is what we will assume here. Under this assumption, the view is an
HTML file that embeds Python code using special {{ }} tags. In particular,
in the example, the {{=message}} instructs web2py to replace the tagged code
with the value of the message returned by the action. Notice that message here
is not a web2py keyword but is defined in the action. So far we have not used
any web2py keywords.
If web2py does not find the requested view, it uses the "generic.html" view
that comes with every application.
If an extension other than "html" is specified ("json" for example), and the
view file "[controller]/[function].json" is not found, web2py looks for the view
"generic.json". web2py comes with generic.html, generic.json, generic.xml,
and generic.rss. These generic views can be modified for each application
individually, and additional views can be added easily.
Generic views are a development tool. In production every action should have
its own view. In fact, by default, generic views are only enabled from localhost.
You can also specify a view with response.view = ’default/something.html’
Read more on this topic in Chapter 10.
If you go back to "EDIT" and click on index, you will now see the following
HTML page:
78 web2py full-stack web framework,4th edition
For debugging purposes you can always append
to the code in a view and it will show you some useful information, including
the request, response and session objects, and list all db queries with their
3.3Let’s count
Let’s now add a counter to this page that will count how many times the
same visitor displays the page. web2py automatically and transparently
tracks visitors using sessions and cookies. For each new visitor, it creates
a session and assigns a unique "session_id". The session is a container for
variables that are stored server-side. The unique id is sent to the browser via
a cookie. When the visitor requests another page from the same application,
the browser sends the cookie back, it is retrieved by web2py, and the
corresponding session is restored.
To use the session, modify the default controller:
1def index():
2if not session.counter:
3session.counter = 1
5session.counter += 1
6return dict(message="Hello from MyApp", counter=session.counter)
Notice that counter is not a web2py keyword but session is. We are asking
web2py to check whether there is a counter variable in the session and, if not,
to create one and set it to 1. If the counter is there, we ask web2py to increase
the counter by 1. Finally we pass the value of the counter to the view.
A more compact way to code the same function is this:
1def index():
2session.counter = (session.counter or 0) + 1
3return dict(message="Hello from MyApp", counter=session.counter)
Now modify the view to add a line that displays the value of the counter:
overview 79
5<h2>Number of visits: {{=counter}}</h2>
When you visit the index page again (and again) you should get the following
HTML page:
The counter is associated with each visitor, and is incremented each time the
visitor reloads the page. Different visitors see different counters.
3.4Say my name
Now create two pages (first and second), where the first page creates a form,
asks the visitor’s name, and redirects to the second page, which greets the
visitor by name.
Write the corresponding actions in the default controller:
1def first():
2return dict()
4def second():
5return dict()
80 web2py full-stack web framework,4th edition
Then create a view "default/first.html" for the first action, and enter:
1{{extend 'layout.html'}}
2What is your name?
3<form action="second">
4<input name="visitor_name" />
5<input type="submit" />
Finally, create a view "default/second.html" for the second action:
1{{extend 'layout.html'}}
2<h1>Hello {{=request.vars.visitor_name}}</h1>
In both views we have extended the basic "layout.html" view that comes with
web2py. The layout view keeps the look and feel of the two pages coherent.
The layout file can be edited and replaced easily, since it mainly contains
HTML code.
If you now visit the first page, type your name:
and submit the form, you will receive a greeting:
overview 81
The mechanism for form submission that we used before is very common, but
it is not good programming practice. All input should be validated and, in
the above example, the burden of validation would fall on the second action.
Thus the action that performs the validation is different from the action that
generated the form. This tends to cause redundancy in the code.
A better pattern for form submission is to submit forms to the same action
that generated them, in our example the "first". The "first" action should
receive the variables, process them, store them server-side, and redirect the
visitor to the "second" page, which retrieves the variables. This mechanism
is called a postback.
Modify the default controller to implement self-submission:
1def first():
82 web2py full-stack web framework,4th edition
2if request.vars.visitor_name:
3session.visitor_name = request.vars.visitor_name
5return dict()
7def second():
8return dict()
Then modify the "default/first.html" view:
1{{extend 'layout.html'}}
2What is your name?
4<input name="visitor_name" />
5<input type="submit" />
and the "default/second.html" view needs to retrieve the data from the
session instead of from the request.vars:
1{{extend 'layout.html'}}
2<h1>Hello {{=session.visitor_name or "anonymous"}}</h1>
From the point of view of the visitor, the self-submission behaves exactly the
same as the previous implementation. We have not added validation yet, but
it is now clear that validation should be performed by the first action.
This approach is better also because the name of the visitor stays in the
session, and can be accessed by all actions and views in the applications
without having to be passed around explicitly.
Note that if the "second" action is ever called before a visitor name is set, it
will display "Hello anonymous" because session.visitor_name returns None.
Alternatively we could have added the following code in the controller
(inside the second function):
1if not request.function=='first' and not session.visitor_name:
This is a general mechanism that you can use to enforce authorization on
controllers, though see Chapter 9for a more powerful method.
With web2py we can move one step further and ask web2py to generate the
form for us, including validation. web2py provides helpers (FORM, INPUT,
overview 83
TEXTAREA, and SELECT/OPTION) with the same names as the equivalent
HTML tags. They can be used to build forms either in the controller or in the
For example, here is one possible way to rewrite the first action:
1def first():
2form = FORM(INPUT(_name='visitor_name', requires=IS_NOT_EMPTY()),
4if form.process().accepted:
5session.visitor_name = form.vars.visitor_name
7return dict(form=form)
where we are saying that the FORM tag contains two INPUT tags. The
attributes of the input tags are specified by the named arguments starting
with underscore. The requires argument is not a tag attribute (because it does
not start by underscore) but it sets a validator for the value of visitor_name.
Here is yet another better wat to create the same form:
1def first():
2form = SQLFORM.factory(Field('visitor_name', requires=IS_NOT_EMPTY()))
3if form.process().accepted:
4session.visitor_name = form.vars.visitor_name
6return dict(form=form)
The form object can be easily serialized in HTML by embedding it in the
"default/first.html" view.
1{{extend 'layout.html'}}
2What is your name?
The form.process() method applies the validators and returns the form itself.
The form.accepted variable is set to True if the form was processed and passed
validation. If the self-submitted form passes validation, it stores the variables
in the session and redirects as before. If the form does not pass validation,
error messages are inserted into the form and shown to the user, as below:
84 web2py full-stack web framework,4th edition
In the next section we will show how forms can be generated automatically
from a model.
3.6An image blog
Here, as another example, we wish to create a web application that allows the
administrator to post images and give them a name, and allows the visitors
of the web site to view the named images and submit comments.
As before, from the site page in admin, create a new application called images,
and navigate to the edit page:
overview 85
We start by creating a model, a representation of the persistent data in the
application (the images to upload, their names, and the comments). First,
you need to create/edit a model file which, for lack of imagination, we call
"db.py". We assume the code below will replace any existing code in "db.py".
Models and controllers must have a .py extension since they are Python code.
If the extension is not provided, it is appended by web2py. Views instead
have a .html extension since they mainly contain HTML code.
Edit the "db.py" file by clicking the corresponding "edit" button:
and enter the following:
1db = DAL("sqlite://storage.sqlite")
4Field('title', unique=True),
86 web2py full-stack web framework,4th edition
6format = '%(title)s')
9Field('image_id', db.image),
10 Field('author'),
11 Field('email'),
12 Field('body','text'))
14 db.image.title.requires = IS_NOT_IN_DB(db, db.image.title)
15 db.comment.image_id.requires = IS_IN_DB(db, db.image.id, '%(title)s')
16 db.comment.author.requires = IS_NOT_EMPTY()
17 db.comment.email.requires = IS_EMAIL()
18 db.comment.body.requires = IS_NOT_EMPTY()
20 db.comment.image_id.writable = db.comment.image_id.readable = False
Let’s analyze this line by line.
Line 1defines a global variable called db that represents the database
connection. In this case it is a connection to a SQLite database stored in
the file "applications/images/databases/storage.sqlite". In the SQLite case,
if the database does not exist, it is created. You can change the name of the
file, as well as the name of the global variable db, but it is convenient to give
them the same name, to make it easy to remember.
Lines 3-5define a table "image". define_table is a method of the db object.
The first argument, "image", is the name of the table we are defining. The
other arguments are the fields belonging to that table. This table has a field
called "title", a field called "file", and a field called "id" that serves as the table
primary key ("id" is not explicitly declared because all tables have an id field
by default). The field "title" is a string, and the field "file" is of type "upload".
"upload" is a special type of field used by the web2py Data Abstraction Layer
(DAL) to store the names of uploaded files. web2py knows how to upload
files (via streaming if they are large), rename them safely, and store them.
When a table is defined, web2py takes one of several possible actions:
if the table does not exist, the table is created;
if the table exists and does not correspond to the definition, the table is
altered accordingly, and if a field has a different type, web2py tries to
overview 87
convert its contents;
if the table exists and corresponds to the definition, web2py does nothing.
This behavior is called "migration". In web2py migrations are automatic, but
can be disabled for each table by passing migrate=False as the last argument
of define_table.
Line 6defines a format string for the table. It determines how a record
should be represented as a string. Notice that the format argument can also
be a function that takes a record and returns a string. For example:
1format=lambda row: row.title
Lines 8-12 define another table called "comment". A comment has an
"author", an "email" (we intend to store the email address of the author of
the comment), a "body" of type "text" (we intend to use it to store the actual
comment posted by the author), and an "image_id" field of type reference
that points to db.image via the "id" field.
In line 14,db.image.title represents the field "title" of table "image". The
attribute requires allows you to set requirements/constraints that will be
enforced by web2py forms. Here we require that the "title" is unique:
IS_NOT_IN_DB(db, db.image.title)
Notice this is optional because it is set automatically given that Field(’title’,
The objects representing these constraints are called validators. Multiple
validators can be grouped in a list. Validators are executed in the order they
appear. IS_NOT_IN_DB(a, b) is a special validator that checks that the value of
a field bfor a new record is not already in a.
Line 15 requires that the field "image_id" of table "comment" is in db.image.id.
As far as the database is concerned, we had already declared this when we
defined the table "comment". Now we are explicitly telling the model that
this condition should be enforced by web2py, too, at the form processing
level when a new comment is posted, so that invalid values do not propagate
from input forms to the database. We also require that the "image_id" be
represented by the "title", ’%(title)s’, of the corresponding record.
88 web2py full-stack web framework,4th edition
Line 20 indicates that the field "image_id" of table "comment" should
not be shown in forms, writable=False and not even in readonly forms,
The meaning of the validators in lines 15-17 should be obvious.
Notice that the validator
1db.comment.image_id.requires = IS_IN_DB(db, db.image.id, '%(title)s')
can be omitted (and would be automatic) if we specify a format for referenced
1db.define_table('image', ..., format='%(title)s')
where the format can be a string or a function that takes a record and returns
a string.
Once a model is defined, if there are no errors, web2py creates an application
administration interface to manage the database. You access it via the
"database administration" link in the edit page or directly:
Here is a screenshot of the appadmin interface:
overview 89
This interface is coded in the controller called "appadmin.py" and the
corresponding view "appadmin.html". From now on, we will refer to this
interface simply as appadmin. It allows the administrator to insert new
database records, edit and delete existing records, browse tables, and perform
database joins.
The first time appadmin is accessed, the model is executed and the tables are
created. The web2py DAL translates Python code into SQL statements that
are specific to the selected database back-end (SQLite in this example). You
can see the generated SQL from the edit page by clicking on the "sql.log" link
under "models". Notice that the link is not present until the tables have been
If you were to edit the model and access appadmin again, web2py would
generate SQL to alter the existing tables. The generated SQL is logged into
Now go back to appadmin and try to insert a new image record:
90 web2py full-stack web framework,4th edition
web2py has translated the db.image.file "upload" field into an upload form
for the file. When the form is submitted and an image file is uploaded, the
file is renamed in a secure way that preserves the extension, it is saved with
the new name under the application "uploads" folder, and the new name is
stored in the db.image.file field. This process is designed to prevent directory
traversal attacks.
Notice that each field type is rendered by a widget. Default widgets can be
When you click on a table name in appadmin, web2py performs a select of
all records on the current table, identified by the DAL query
1db.image.id > 0
and renders the result.
overview 91
You can select a different set of records by editing the SQL query and pressing
To edit or delete a single record, click on the record id number.
Because of the IS_IN_DB validator, the reference field "image_id" is rendered
by a drop-down menu. The items in the drop-down are stored as keys
(db.image.id), but are represented by their db.image.title, as specified by
the validator.
Validators are powerful objects that know how to represent fields, filter field
values, generate errors, and format values extracted from the field.
The following figure shows what happens when you submit a form that does
not pass validation:
92 web2py full-stack web framework,4th edition
The same forms that are automatically generated by appadmin can also be
generated programmatically via the SQLFORM helper and embedded in user
applications. These forms are CSS-friendly, and can be customized.
Every application has its own appadmin; therefore, appadmin itself can be
modified without affecting other applications.
So far, the application knows how to store data, and we have seen how to
access the database via appadmin. Access to appadmin is restricted to the
administrator, and it is not intended as a production web interface for the
application; hence the next part of this walk-through. Specifically we want to
An "index" page that lists all available images sorted by title and links to
detail pages for the images.
A "show/[id]" page that shows the visitor the requested image and allows
the visitor to view and post comments.
A "download/[name]" action to download uploaded images.
This is represented schematically here:
overview 93
Go back to the edit page and edit the "default.py" controller, replacing its
contents with the following:
1def index():
2images = db().select(db.image.ALL, orderby=db.image.title)
3return dict(images=images)
This action returns a dictionary. The keys of the items in the dictionary are
interpreted as variables passed to the view associated to the action. When
developing, if there is no view, the action is rendered by the "generic.html"
view that is provided with every web2py application.
The index action performs a select of all fields (db.image.ALL) from table
image, ordered by db.image.title. The result of the select is a Rows object
containing the records. Assign it to a local variable called images returned by
the action to the view. images is iterable and its elements are the selected rows.
For each row the columns can be accessed as dictionaries: images[0][’title’]
or equivalently as images[0].title.
If you do not write a view, the dictionary is rendered by "views/generic.html"
and a call to the index action would look like this:
You have not created a view for this action yet, so web2py renders the set of
records in plain tabular form.
Proceed to create a view for the index action. Return to admin, edit
"default/index.html" and replace its content with the following:
94 web2py full-stack web framework,4th edition
1{{extend 'layout.html'}}
2<h1>Current Images</h1>
4{{for image in images:}}
5{{=LI(A(image.title, _href=URL("show", args=image.id)))}}
The first thing to notice is that a view is pure HTML with special {{...}} tags.
The code embedded in {{...}} is pure Python code with one caveat: indentation
is irrelevant. Blocks of code start with lines ending in colon (:) and end in
lines beginning with the keyword pass. In some cases the end of a block is
obvious from context and the use of pass is not required.
Lines 5-7loop over the image rows and for each row image display:
1LI(A(image.title, _href=URL('show', args=image.id))
This is a <li>...</li> tag that contains an <a href="...">...</a> tag which
contains the image.title. The value of the hypertext reference (href attribute)
1URL('show', args=image.id)
i.e., the URL within the same application and controller as the current
request that calls the function called "show", passing a single argument to
the function, args=image.id.LI,A, etc. are web2py helpers that map to
the corresponding HTML tags. Their unnamed arguments are interpreted
as objects to be serialized and inserted in the tag’s innerHTML. Named
arguments starting with an underscore (for example _href) are interpreted
as tag attributes but without the underscore. For example _href is the href
attribute, _class is the class attribute, etc.
As an example, the following statement:
1{{=LI(A('something',_href=URL('show', args=123))}}
is rendered as:
1<li><a href="/images/default/show/123">something</a></li>
A handful of helpers (INPUT,TEXTAREA,OPTION and SELECT) also support some
overview 95
special named attributes not starting with underscore (value, and requires).
They are important for building custom forms and will be discussed later.
Go back to the edit page. It now indicates that "default.py exposes index". By
clicking on "index", you can visit the newly created page:
which looks like:
If you click on the image name link, you are directed to:
and this results in an error, since you have not yet created an action called
"show" in controller "default.py".
Let’s edit the "default.py" controller and replace its content with:
1def index():
2images = db().select(db.image.ALL, orderby=db.image.title)
3return dict(images=images)
5def show():
6image = db(db.image.id==request.args(0)).select().first()
7db.comment.image_id.default = image.id
8form = SQLFORM(db.comment)
9if form.process().accepted:
10 response.flash = 'your comment is posted'
11 comments = db(db.comment.image_id==image.id).select()
12 return dict(image=image, comments=comments, form=form)
14 def download():
15 return response.download(request, db)
96 web2py full-stack web framework,4th edition
The controller contains two actions: "show" and "download". The "show"
action selects the image with the id parsed from the request args and all
comments related to the image. "show" then passes everything to the view
The image id referenced by:
1URL('show', args=image.id)
in "default/index.html", can be accessed as: request.args(0) from the "show"
The "download" action expects a filename in request.args(0), builds a path
to the location where that file is supposed to be, and sends it back to the
client. If the file is too large, it streams the file without incurring any memory
Notice the following statements:
Line 7creates an insert form SQLFORM for the db.comment table using only
the specified fields.
Line 8sets the value for the reference field, which is not part of the input
form because it is not in the list of fields specified above.
Line 9processes the submitted form (the submitted form variables are in
request.vars) within the current session (the session is used to prevent
double submissions, and to enforce navigation). If the submitted form
variables are validated, the new comment is inserted in the db.comment
table; otherwise the form is modified to include error messages (for
example, if the author’s email address is invalid). This is all done in line
Line 10 is only executed if the form is accepted, after the record is
inserted into the database table. response.flash is a web2py variable that
is displayed in the views and used to notify the visitor that something
Line 11 selects all comments that reference the current image.
The "download" action is already defined in the "default.py" controller of the
overview 97
scaffolding application.
The "download" action does not return a dictionary, so it does not need a
view. The "show" action, though, should have a view, so return to admin
and create a new view called "default/show.html".
Edit this new file and replace its content with the following:
1{{extend 'layout.html'}}
2<h1>Image: {{=image.title}}</h1>
4<img width="200px"
5src="{{=URL('download', args=image.file)}}" />
7{{if len(comments):}}
8<h2>Comments</h2><br /><p>
9{{for comment in comments:}}
10 <p>{{=comment.author}} says <i>{{=comment.body}}</i></p>
11 {{pass}}</p>
12 {{else:}}
13 <h2>No comments posted yet</h2>
14 {{pass}}
15 <h2>Post a comment</h2>
16 {{=form}}
This view displays the image.file by calling the "download" action inside an
<img ... /> tag. If there are comments, it loops over them and displays each
Here is how everything will appear to a visitor.
98 web2py full-stack web framework,4th edition
When a visitor submits a comment via this page, the comment is stored in
the database and appended at the bottom of the page.
3.7Adding CRUD
web2py also provides a CRUD (Create/Read/Update/Delete) API that
simplifies forms even more. To use CRUD it is necessary to define it
somewhere, such as in file "db.py":
1from gluon.tools import Crud
2crud = Crud(db)
These two lines are already in the scaffolding application.
The crud object provides high-level methods, for example:
1form = crud.create(table)
that can be used to replace the programming pattern:
1form = SQLFORM(table)
2if form.process().accepted:
3session.flash = '...'
overview 99
Here, we rewrite the previous "show" action using crud and making some
more improvements:
1def show():
2image = db.image(request.args(0)) or redirect(URL('index'))
3db.comment.image_id.default = image.id
4form = crud.create(db.comment,
5message='your comment is posted',
7comments = db(db.comment.image_id==image.id).select()
8return dict(image=image, comments=comments, form=form)
First of all notice we have used the syntax
1db.image(request.args(0)) or redirect(...)
to fetch the required record. Since ‘table(id) returns None if the record is
not found, we can use or redirect(...) in this case in one line.
The next argument of crud.create is the URL to redirect to after the form is
accepted. The message argument is the one to be displayed upon acceptance.
You can read more about CRUD in Chapter 7.
3.8Adding Authentication
The web2py API for Role-Based Access Control is quite sophisticated, but
for now we will limit ourselves to restricting access to the show action to
authenticated users, deferring a more detailed discussion to Chapter 9.
To limit access to authenticated users, we need to complete three steps. In a
model, for example "db.py", we need to add:
1from gluon.tools import Auth
2auth = Auth(db)
In our controller, we need to add one action:
1def user():
2return dict(form=auth())
100 web2py full-stack web framework,4th edition
This is sufficient to enable login, register, logout, etc. pages. The default
layout will also show options to the corresponding pages in the top right
We can now decorate the functions that we want to restrict, for example:
2def show():
3image = db.image(request.args(0)) or redirect(URL('index'))
4db.comment.image_id.default = image.id
5form = crud.create(db.comment, next=URL(args=image.id),
6message='your comment is posted')
7comments = db(db.comment.image_id==image.id).select()
8return dict(image=image, comments=comments, form=form)
Any attempt to access
will require login. If the user is not logged it, the user will be redirected to
overview 101
The user function also exposes, among others, the following actions:
Now, a first-time user needs to register in order to be able to log in and read
or post comments.
Both the auth object and the user function are already defined in the scaffolding
application. The auth object is highly customizable and can deal with email
verification, registration approvals, CAPTCHA, and alternate login methods
via plugins.
102 web2py full-stack web framework,4th edition
3.8.1Adding grids
We can improve this further using the SQLFORM.grid and SQLFORM.smartgrid
gadgets to create a management interface for our application:
2def manage():
3grid = SQLFORM.smartgrid(db.image)
4return dict(grid=grid)
with associated "views/default/manage.html"
1{{extend 'layout.html'}}
2<h2>Management Interface</h2>
Using appadmin create a group "manager" and make some users members
of the group. They will not be able to access
and browse, search:
create, update and delete images and their comments:
overview 103
3.9Configuring the layout
You can configure the default layout by editing "views/layout.html" but you
can also configure it without editing the HTML. In fact, the "static/base.css"
stylesheet is well documented and described in Chapter 5. You can change
color, columns, size, borders and background without editing the HTML. If
you want to edit the menu, the title or the subtitle, you can do so in any
model file. The scaffolding app, sets default values of these parameters in
the file "models/menu.py":
1response.title = request.application
2response.subtitle = T('customize me!')
3response.meta.author = 'you'
4response.meta.description = 'describe your app'
5response.meta.keywords = 'bla bla bla'
6response.menu = [ [ 'Index', False, URL('index')]]
104 web2py full-stack web framework,4th edition
3.10 A wiki
In this section, we build a wiki, from scratch and without using the extended
functionality provided by plugin_wiki which is described in chapter 12. The
visitor will be able to create pages, search them (by title), and edit them.
The visitor will also be able to post comments (exactly as in the previous
applications), and also post documents (as attachments to the pages) and
link them from the pages. As a convention, we adopt the Markmin syntax
for our wiki syntax. We will also implement a search page with Ajax, an RSS
feed for the pages, and a handler to search the pages via XML-RPC [46].
The following diagram lists the actions that we need to implement and the
links we intend to build among them.
overview 105
Start by creating a new scaffolding app, naming it "mywiki".
The model must contain three tables: page, comment, and document. Both
comment and document reference page because they belong to page. A
document contains a file field of type upload as in the previous images
Here is the complete model:
1db = DAL('sqlite://storage.sqlite')
3from gluon.tools import *
4auth = Auth(db)
6crud = Crud(db)
10 Field('body','text'),
11 Field('created_on','datetime', default=request.now),
12 Field('created_by', db.auth_user, default=auth.user_id),
13 format='%(title)s')
15 db.define_table('comment',
16 Field('page_id', db.page),
17 Field('body','text'),
18 Field('created_on','datetime', default=request.now),
19 Field('created_by', db.auth_user, default=auth.user_id))
21 db.define_table('document',
22 Field('page_id', db.page),
23 Field('name'),
24 Field('file','upload'),
25 Field('created_on','datetime', default=request.now),
26 Field('created_by', db.auth_user, default=auth.user_id),
27 format='%(name)s')
29 db.page.title.requires = IS_NOT_IN_DB(db, 'page.title')
30 db.page.body.requires = IS_NOT_EMPTY()
31 db.page.created_by.readable = db.page.created_by.writable = False
32 db.page.created_on.readable = db.page.created_on.writable = False
34 db.comment.body.requires = IS_NOT_EMPTY()
35 db.comment.page_id.readable = db.comment.page_id.writable = False
36 db.comment.created_by.readable = db.comment.created_by.writable = False
37 db.comment.created_on.readable = db.comment.created_on.writable = False
39 db.document.name.requires = IS_NOT_IN_DB(db, 'document.name')
106 web2py full-stack web framework,4th edition
40 db.document.page_id.readable = db.document.page_id.writable = False
41 db.document.created_by.readable = db.document.created_by.writable = False
42 db.document.created_on.readable = db.document.created_on.writable = False
Edit the controller "default.py" and create the following actions:
index: list all wiki pages
create: post another wiki page
show: show a wiki page and its comments, and append comments
edit: edit an existing page
documents: manage the documents attached to a page
download: download a document (as in the images example)
search: display a search box and, via an Ajax callback, return all matching
titles as the visitor types
callback: the Ajax callback function. It returns the HTML that gets
embedded in the search page while the visitor types.
Here is the "default.py" controller:
1def index():
2""" this controller returns a dictionary rendered by the view
3it lists all wiki pages
4>>> index().has_key('pages')
7pages = db().select(db.page.id,db.page.title,orderby=db.page.title)
8return dict(pages=pages)
10 @auth.requires_login()
11 def create():
12 "creates a new empty wiki page"
13 form = crud.create(db.page, next=URL('index'))
14 return dict(form=form)
16 def show():
17 "shows a wiki page"
18 this_page = db.page(request.args(0)) or redirect(URL('index'))
19 db.comment.page_id.default = this_page.id
20 form = crud.create(db.comment) if auth.user else None
21 pagecomments = db(db.comment.page_id==this_page.id).select()
22 return dict(page=this_page, comments=pagecomments, form=form)
overview 107
24 @auth.requires_login()
25 def edit():
26 "edit an existing wiki page"
27 this_page = db.page(request.args(0)) or redirect(URL('index'))
28 form = crud.update(db.page, this_page,
29 next=URL('show',args=request.args))
30 return dict(form=form)
32 @auth.requires_login()
33 def documents():
34 "browser, edit all documents attached to a certain page"
35 page = db.page(request.args(0)) or redirect(URL('index'))
36 db.document.page_id.default = page.id
37 db.document.page_id.writable = False
38 grid = SQLFORM.grid(db.document.page_id==page.id,args=[page.id])
39 return dict(page=page, grid=grid)
41 def user():
42 return dict(form=auth())
44 def download():
45 "allows downloading of documents"
46 return response.download(request, db)
48 def search():
49 "an ajax wiki search page"
50 return dict(form=FORM(INPUT(_id='keyword',_name='keyword',
51 _onkeyup="ajax('callback', ['keyword'], 'target');")),
52 target_div=DIV(_id='target'))
54 def callback():
55 "an ajax callback that returns a <ul> of links to wiki pages"
56 query = db.page.title.contains(request.vars.keyword)
57 pages = db(query).select(orderby=db.page.title)
58 links = [A(p.title, _href=URL('show',args=p.id)) for p in pages]
59 return UL(*links)
Lines 2-6provide a comment for the index action. Lines 4-5inside the
comment are interpreted by python as test code (doctest). Tests can be run
via the admin interface. In this case the tests verify that the index action runs
without errors.
Lines 18,27, and 35 try to fetch a page record with the id in request.args(0).
Lines 13,20 define and process create forms for a new page and a new
comment and.
108 web2py full-stack web framework,4th edition
Line 28 defines and processes an update form for a wiki page.
Line 38 creates a grid object that allows to browser, add and update the
comments linked to a page.
Some magic happens in line 51. The onkeyup attribute of the INPUT tag
"keyword" is set. Every time the visitor releases a key, the JavaScript code
inside the onkeyup attribute is executed, client-side. Here is the JavaScript
1ajax('callback', ['keyword'], 'target');
ajax is a JavaScript function defined in the file "web2py.js" which is included
by the default "layout.html". It takes three parameters: the URL of the action
that performs the synchronous callback, a list of the IDs of variables to be
sent to the callback (["keyword"]), and the ID where the response has to be
inserted ("target").
As soon as you type something in the search box and release a key, the
client calls the server and sends the content of the ’keyword’ field, and,
when the sever responds, the response is embedded in the page itself as
the innerHTML of the ’target’ tag.
The ’target’ tag is a DIV defined in line 52. It could have been defined in the
view as well.
Here is the code for the view "default/create.html":
1{{extend 'layout.html'}}
2<h1>Create new wiki page</h1>
If you visit the create page, you see the following:
overview 109
Here is the code for the view "default/index.html":
1{{extend 'layout.html'}}
2<h1>Available wiki pages</h1>
3[ {{=A('search',_href=URL('search'))}} ]<br />
4<ul>{{for page in pages:}}
5{{=LI(A(page.title, _href=URL('show', args=page.id)))}}
7[ {{=A('create page',_href=URL('create'))}} ]
It generates the following page:
110 web2py full-stack web framework,4th edition
Here is the code for the view "default/show.html":
1{{extend 'layout.html'}}
3[ {{=A('edit',_href=URL('edit', args=request.args))}}
4| {{=A('documents',_href=URL('documents', args=request.args))}} ]<br />
7{{for comment in comments:}}
8<p>{{=db.auth_user[comment.created_by].first_name}} on {{=comment.created_on}}
9says <I>{{=comment.body}}</i></p>
10 {{pass}}
11 <h2>Post a comment</h2>
12 {{=form}}
If you wish to use markdown syntax instead of markmin syntax:
1from gluon.contrib.markdown import WIKI
and use WIKI instead of the MARKMIN helper. Alternatively, you can choose to
accept raw HTML instead of markmin syntax. In this case you would replace:
overview 111
(so that the XML does not get escaped, as by default web2py behavior).
This can be done better with:
1{{=XML(page.body, sanitize=True)}}
By setting sanitize=True, you tell web2py to escape unsafe XML tags such as
"<script>", and thus prevent XSS vulnerabilities.
Now if, from the index page, you click on a page title, you can see the page
that you have created:
Here is the code for the view "default/edit.html":
1{{extend 'layout.html'}}
2<h1>Edit wiki page</h1>
3[ {{=A('show',_href=URL('show', args=request.args))}} ]<br />
It generates a page that looks almost identical to the create page.
Here is the code for the view "default/documents.html":
112 web2py full-stack web framework,4th edition
1{{extend 'layout.html'}}
2<h1>Documents for page: {{=page.title}}</h1>
3[ {{=A('show',_href=URL('show', args=request.args))}} ]<br />
If, from the "show" page, you click on documents, you can now manage the
documents attached to the page.
Finally here is the code for the view "default/search.html":
1{{extend 'layout.html'}}
2<h1>Search wiki pages</h1>
3[ {{=A('listall',_href=URL('index'))}}]<br />
4{{=form}}<br />{{=target_div}}
which generates the following Ajax search form:
overview 113
You can also try to call the callback action directly by visiting, for example,
the following URL:
If you look at the page source you see the HTML returned by the callback:
1<ul><li><a href="/mywiki/default/show/4">I made a Wiki</a></li></ul>
Generating an RSS feed from the stored pages using web2py is easy because
web2py includes gluon.contrib.rss2. Just append the following action to the
default controller:
1def news():
2"generates rss feed form the wiki pages"
3reponse.generic_patterns = ['.rss']
4pages = db().select(db.page.ALL, orderby=db.page.title)
5return dict(
6title = 'mywiki rss feed',
7link = '',
8description = 'mywiki news',
9created_on = request.now,
10 items = [
11 dict(title = row.title,
12 link = URL('show', args=row.id),
13 description = MARKMIN(row.body).xml(),
14 created_on = row.created_on
15 ) for row in pages])
114 web2py full-stack web framework,4th edition
and when you visit the page
you see the feed (the exact output depends on the feed reader). Notice that
the dict is automatically converted to RSS, thanks to the.rss extension in the
web2py also includes feedparser to read third-party feeds.
Finally, let’s add an XML-RPC handler that allows searching the wiki
1service = Service()
4def find_by(keyword):
5"finds pages that contain keyword for XML-RPC"
6return db(db.page.title.contains(keyword).select().as_list()
8def call():
9"exposes all registered services, including XML-RPC"
10 return service()
Here, the handler action simply publishes (via XML-RPC), the functions
specified in the list. In this case, find_by.find_by is not an action (because it
takes an argument). It queries the database with .select() and then extracts
the records as a list with .response and returns the list.
overview 115
Here is an example of how to access the XML-RPC handler from an external
Python program.
1>>> import xmlrpclib
2>>> server = xmlrpclib.ServerProxy(
4>>> for item in server.find_by('wiki'):
5print item['created_on'], item['title']
The handler can be accessed from many other programming languages that
understand XML-RPC, including C, C++, C# and Java.
3.10.1On date,datetime and time format
There are three different representation for each of the field types date,
datetime and time:
the database representation
the internal web2py prepresentation
the string representation in forms and tables
The database representation is an internal issue and does not affect the
code. Internally, at the web2py level, they are stored as datetime.date,
datetime.datetime and datetime.time object respectively and they can be
manipulated as such:
1for page in db(db.page).select():
2print page.title, page.day, page.month, page.year
When dates are converted to strings in forms they are converted using the
ISO representation
1%Y-%m-%d %H:%M:%S
yet this representation in internationalized and you can use the admin
stranslation page to change the format to an alternate one. For example:
1%m/%b/%Y %H:%M:%S
Mind that by default English is not translated because web2py assumes the
applications is already written in English. If you want internationalization
116 web2py full-stack web framework,4th edition
to work for English you need to create the translation file (using admin) and
you need declare that the application current language is something other than
english, for example:
1T.current_languages = ['null']
3.11 More on admin
The administrative interface provides additional functionality that we briefly
review here.
This page lists all installed applications. There are two forms at the bottom.
The first of them allows creating a new application by specifying its name.
The second form allows uploading an existing application from either a local
file or a remote URL. When you upload an application, you need to specify
a name for it. This can be its original name, but does not need to be. This
allows installing multiple copies of the same application. You can try, for
example, to upload the the Instant Press CMS created by Martin Mulone
Web2py files are packages as .w2p files. These ones are tar gzipped files.
Web2py uses the .w2p extension instead of the .tgz extension to prevent the
browser from unzipping on download. They can be uncompressed manually
with tar zxvf [filename] although this is never necessary.
overview 117
Upon successful upload, web2py displays the MD5checksum of the
uploaded file. You can use it to verify that the file was not corrupted
during upload. The InstantPress name will appear in the list of installed
Click on the InstantPress name on admin to get it up and running.
You can read more about Instant Press at the following URL:
For each application the site page allows you to:
Uninstall the application.
Jump to the about page (read below).
118 web2py full-stack web framework,4th edition
Jump to the edit page (read below).
Jump to the errors page (read below).
Clean up temporary files (sessions, errors, and cache.disk files).
• Pack all. This returns a tar file containing a complete copy of the
application. We suggest that you clean up temporary files before packing
an application.
Compile the application. If there are no errors, this option will bytecode-
compile all models, controllers and views. Because views can extend and
include other views in a tree, before bytecode compilation, the view tree
for every controller is collapsed into a single file. The net effect is that a
bytecode-compiled application is faster, because there is no more parsing
of templates or string substitutions occurring at runtime.
Pack compiled. This option is only present for bytecode-compiled
applications. It allows packing the application without source code
for distribution as closed source. Note that Python (as any other
programming language) can technically be decompiled; therefore
compilation does not provide complete protection of the source code.
Nevertheless, decompilation can be difficult and can be illegal.
Remove compiled. It simply removes the byte-code compiled models,
views and controllers from the application. If the application was
packaged with source code or edited locally, there is no harm in removing
the bytecode-compiled files, and the application will continue to work. If
the application was installed form a packed compiled file, then this is not
safe, because there is no source code to revert to, and the application will
no longer work.
All the functionality available from the web2py admin site page is also
accessible programmatically via the API defined in the module gluon/admin.py.
Simply open a python shell and import this module.
overview 119
The about tab allows editing the description of the application and its license.
These are written respectively in the ABOUT and LICENSE files in the
application folder.
You can use MARKMIN, or gluon.contrib.markdown.WIKI syntax for these files as
described in ref. [29].
You have used the edit page already in this chapter. Here we want to point
out a few more functionalities of the edit page.
If you click on any file name, you can see the contents of the file with
syntax highlighting.
If you click on edit, you can edit the file via a web interface.
If you click on delete, you can delete the file (permanently).
If you click on test, web2py will run tests. Tests are written by the
developer using Python doctests, and each function should have its own
You can add language files, scan the app to discover all strings, and edit
120 web2py full-stack web framework,4th edition
string translations via the web interface.
If the static files are organized in folders and subfolders, the folder
hierarchy can be toggled by clicking on a folder name.
The image below shows the output of the test page for the welcome
The image below show the languages tab for the welcome application.
overview 121
The image below shows how to edit a language file, in this case the "it"
(Italian) language for the welcome application.
122 web2py full-stack web framework,4th edition
If you click on the "shell" link under the controllers tab in edit, web2py will
open a web based Python shell and will execute the models for the current
application. This allows you to interactively talk to your application.
Also under the controllers tab in edit there is a "crontab" link. By clicking
on this link you will be able to edit the web2py crontab file. This follows
the same syntax as the unix crontab but does not rely on unix. In fact, it
only requires web2py, and it works on Windows. It allows you to register
actions that need to be executed in background at scheduled times. For more
information about this, see the next chapter.
When programming web2py, you will inevitably make mistakes and
introduce bugs. web2py helps in two ways: 1) it allows you to create tests
for every function that can be run in the browser from the edit page; and 2)
when an error manifests itself, a ticket is issued to the visitor and the error is
Intentionally introduce an error in the images application as shown below:
overview 123
1def index():
2images = db().select(db.image.ALL,orderby=db.image.title)
4return dict(images=images)
When you access the index action, you get the following ticket:
Only the administrator can access the ticket:
The ticket shows the traceback, and the content of the file that caused the
problem, and the complete state of system (variables, request, session, etc.)
If the error occurs in a view, web2py shows the view converted from HTML
into Python code. This allows to easily identify the logical structure of the
124 web2py full-stack web framework,4th edition
By default tickets are stored on filesystem and group by traceback. The
administrative interface provides an aggregate views (type of traceback and
number of occurrence) and a detailed view (all tickets are listed by ticket id).
The administrator can switch between the two views.
Notice that everywhere admin shows syntax-highlighted code (for example,
in error reports, web2py keywords are shown in orange). If you click on
a web2py keyword, you are redirected to a documentation page about the
If you fix the divide-by-zero bug in the index action and introduce one in the
index view:
1{{extend 'layout.html'}}
3<h1>Current Images</h1>
5{{for image in images:}}
7{{=LI(A(image.title, _href=URL("show", args=image.id)))}}
you get the following ticket:
Note that web2py has converted the view from HTML into a Python file, and
overview 125
the error described in the ticket refers to the generated Python code and NOT
to the original view file:
This may seem confusing at first, but in practice it makes debugging easier,
because the Python indentation highlights the logical structure of the code
that you embedded in the views.
The code is shown at the bottom of the same page.
All tickets are listed under admin in the errors page for each application:
126 web2py full-stack web framework,4th edition
If you are running from source and you have the Mercurial version control
libraries installed:
1easy_install mercurial
then the administrative interface shows one more menu item called
"mercurial". It automatically creates a local Mercurial repository for the
application. Pressing the "commit" button in the page will commit the current
application. Mercurial creates and stores information about changes you
make in your code into a hidden folder ".hg" in your app subfolder. Every
app has its own ".hg" folder and its own ".hgignore" file (tells Mercurial which
files to ignore).
The Mercurial web interface does allow you to browse previous commit and
diff files but we do recommend you use Mercurial directly from the shell or
one of the may GUI-based Mercurial clients since they are more powerful. For
example they will allow you sync your app with a remote source repository:
overview 127
You can read more about Mercurial here:
3.11.6Admin wizard (experimental)
The admin interface includes a Wizard that can help you create a new
applications. You can access the wizard from the "sites" page as shown in
the image below.
The wizard will guide you through a series of steps involved in creating a
new application:
Chose a name for the application
128 web2py full-stack web framework,4th edition
Configure the application and choose required plugins
Build required models (it will create CRUD pages for each model)
Allow you to edit the views of those pages using MARKMIN syntax
The image below shows the second step of the process.
You can see a dropdown to select a layout plugin (from web2py.com/layouts),
a multiple choice dropdown to check other plugins (from web2py.com/plugins)
and a "login config" field where to put the Janrain "domain:key".
The other steps are pretty much self-explanatory.
The Wizard works well for what it does but it is considered an experimental
feature for two reasons:
Applications created with the wizard and edited manually, cannot later be
modified by the wizard.
The interface of the wizard will change over time to include support for
more features and easier visual development.
In any case the wizard is a handy tool for fast prototyping and it can be used
to bootstrap a new application with an alternate layout and optional plugins.
overview 129
3.11.7Configuring admin
Normally there is no need to perform any configuration of admin but a few
customizations are possible. After you login into admin you can edit the
admin configuration file via the URL:
Notice that admin can be used to edit itself. In fact admin is an app as any
other one.
The file "0.py" is very much self documented and if you are opening
probably you already know what you are looking for. Anyway there a few
customizations that are more important than others:
1GAE_APPCFG = os.path.abspath(os.path.join('/usr/local/bin/appcfg.py'))
This should point to the location of the "appcfg.py" file that comes with the
Google App Engine SDK. If you have the SDK you may want to change this
config parameters to the correct value. It will allow you to deploy to GAE
from the admin interface.
You can also set web2py admin in demo mode:
2FILTER_APPS = ['welcome']
And only the apps listed in filter apps will be accessible and they will be only
accessible in read-only mode.
If you are a teacher and want to expose the administrative interface to
students so that students can share one administrative interface for their
projects (think of a virtual lab), can do it by setting:
In this way students will be required to login and will only be able to access
their own apps via admin. You, as first user/teacher, will be able to access
them all.
Mind that this mechanism still assumes all users are trusted. All the apps
created under admin run under the same credentials on the same filesystem.
130 web2py full-stack web framework,4th edition
It is possible for an app created by a student to access the data and the source
of an app created by another student.
3.12 More on appadmin
appadmin is not intended to be exposed to the public. It is designed to help
you by providing an easy access to the database. It consists of only two files:
a controller "appadmin.py" and a view "appadmin.html" which are used by
all actions in the controller.
The appadmin controller is relatively small and readable; it provides an
example of designing a database interface.
appadmin shows which databases are available and which tables exist in
each database. You can insert records and list all records for each table
individually. appadmin paginates output 100 records at a time.
Once a set of records is selected, the header of the pages changes, allowing
you to update or delete the selected records.
To update the records, enter an SQL assignment in the Query string field:
1title = 'test'
where string values must be enclosed in single quotes. Multiple fields can be
separated by commas.
To delete a record, click the corresponding checkbox to confirm that you are
appadmin can also perform joins if the SQL FILTER contains a SQL condition
that involves two or more tables. For example, try:
1db.image.id == db.comment.image_id
web2py passes this along to the DAL, and it understands that the query links
two tables; hence, both tables are selected with an INNER JOIN. Here is the
overview 131
If you click on the number of an id field, you get an edit page for the record
with the corresponding id.
If you click on the number of a reference field, you get an edit page for the
referenced record.
You cannot update or delete rows selected by a join, because they involve
records from multiple tables and this would be ambiguous.
In addition to its database administration capabilities, appadmin also
enables you to view details about the contents of the application’s cache
(at /yourapp/appadmin/ccache) as well as the contents of the current request,
response, and session objects (at /yourapp/appadmin/state).
appadmin replaces response.menu with its own menu, which provides links
to the application’s edit page in admin, the db (database administration)
page, the state page, and the cache page. If your application’s layout
does not generate a menu using response.menu, then you will not see the
appadmin menu. In that case, you can modify the appadmin.html file and
add {{=MENU(response.menu)}} to display the menu.
The core
4.1Command line options
It is possible to skip the GUI and start web2py directly from the command
line by typing something like:
1python web2py.py -a 'your password' -i -p 8000
When web2py starts, it creates a file called "parameters_8000.py" where it
stores the hashed password. If you use "<ask>" as the password, web2py
prompts you for it.
For additional security, you can start web2py with:
1python web2py.py -a '<recycle>' -i -p 8000
In this case web2py reuses the previously stored hashed password. If no
password is provided, or if the "parameters_8000.py" file is deleted, the web-
based administrative interface is disabled.
On some Unix/Linux systems, if the password is
web2py uses the PAM password of the Operating System account of
some_user to authenticate the administrator, unless blocked by the PAM
134 web2py full-stack web framework,4th edition
web2py normally runs with CPython (the C implementation of the Python
interpreter created by Guido van Rossum), but it can also run with Jython
(the Java implementation of the interpreter). The latter possibility allows
the use of web2py in the context of a J2EE infrastructure. To use Jython,
simply replace "python web2py.py..." with "jython web2py.py". Details about
installing Jython, zxJDBC modules required to access the databases can be
found in Chapter 14.
The "web2py.py" script can take many command-line arguments specifying
the maximum number of threads, enabling of SSL, etc. For a complete list
1>>> python web2py.py -h
2Usage: python web2py.py
4web2py Web Framework startup script. ATTENTION: unless a password
5is specified (-a 'passwd'), web2py will attempt to run a GUI.
6In this case command line options are ignored.
9--version show program's version number and exit
10 -h, --help show this help message and exit
11 -i IP, --ip=IP ip address of the server (
12 -p PORT, --port=PORT port of server (8000)
13 -a PASSWORD, --password=PASSWORD
14 password to be used for administration (use -a
15 "<recycle>" to reuse the last password))
16 -c SSL_CERTIFICATE, --ssl_certificate=SSL_CERTIFICATE
17 file that contains ssl certificate
18 -k SSL_PRIVATE_KEY, --ssl_private_key=SSL_PRIVATE_KEY
19 file that contains ssl private key
20 -d PID_FILENAME, --pid_filename=PID_FILENAME
21 file to store the pid of the server
22 -l LOG_FILENAME, --log_filename=LOG_FILENAME
23 file to log connections
24 -n NUMTHREADS, --numthreads=NUMTHREADS
25 number of threads (deprecated)
26 --minthreads=MINTHREADS
27 minimum number of server threads
28 --maxthreads=MAXTHREADS
29 maximum number of server threads
30 -s SERVER_NAME, --server_name=SERVER_NAME
31 server name for the web server
32 -q REQUEST_QUEUE_SIZE, --request_queue_size=REQUEST_QUEUE_SIZE
33 max number of queued requests when server unavailable
34 -o TIMEOUT, --timeout=TIMEOUT
the core 135
35 timeout for individual request (10 seconds)
37 timeout on shutdown of server (5 seconds)
38 -f FOLDER, --folder=FOLDER
39 folder from which to run web2py
40 -v, --verbose increase --test verbosity
41 -Q, --quiet disable all output
43 set debug output level (0-100, 0 means all, 100 means
44 none; default is 30)
45 -S APPNAME, --shell=APPNAME
46 run web2py in interactive shell or IPython (if
47 installed) with specified appname (if app does not
48 exist it will be created). APPNAME like a/c/f (c,f
49 optional)
50 -B, --bpython run web2py in interactive shell or bpython (if
51 installed) with specified appname (if app does not
52 exist it will be created). Use combined with --shell
53 -P, --plain only use plain python shell; should be used with
54 --shell option
55 -M, --import_models auto import model files; default is False; should be
56 used with --shell option
58 run PYTHON_FILE in web2py environment; should be used
59 with --shell option
60 -K SCHEDULER, --scheduler=SCHEDULER
61 run scheduled tasks for the specified apps
62 -K app1, app2, app3 requires a scheduler defined in the
63 models of the respective apps
65 run doctests in web2py environment; TEST_PATH like
66 a/c/f (c,f optional)
68 -W install|start|stop as Windows service
69 -C, --cron trigger a cron run manually; usually invoked from a
70 system crontab
71 --softcron triggers the use of softcron
72 -N, --no-cron do not start cron automatically
73 -J, --cronjob identify cron-initiated command
74 -L CONFIG, --config=CONFIG
75 config file
77 profiler filename
78 -t, --taskbar use web2py gui and run in taskbar (system tray)
79 --nogui text-only, no GUI
80 -A ARGS, --args=ARGS should be followed by a list of arguments to be passed
81 to script, to be used with -S, -A must be the last
82 option
83 --no-banner Do not print header banner
136 web2py full-stack web framework,4th edition
84 --interfaces=INTERFACES
85 listen on multiple addresses:
86 "ip:port:cert:key;ip2:port2:cert2:key2;..." (:cert:key
87 optional; no spaces)
Lower-case options are used to configure the web server. The -L option tells
web2py to read configuration options from a file, -W installs web2py as a
windows service, while -S,-P and -M options start an interactive Python
shell. The -T option finds and runs controller doctests in a web2py execution
environment. For example, the following example runs doctests from all
controllers in the "welcome" application:
1python web2py.py -vT welcome
if you run web2py as Windows Service, -W, it is not convenient to pass the
configuration using command line arguments. For this reason, in the web2py
folder there is a sample "options_std.py" configuration file for the internal
web server:
1import socket
2import os
4ip = ''
5port = 80
8password = '<recycle>' # ## <recycle> means use the previous password
9pid_filename = 'httpserver.pid'
10 log_filename = 'httpserver.log'
11 profiler_filename = None
12 minthreads = None
13 maxthreads = None
14 server_name = socket.gethostname()
15 request_queue_size = 5
16 timeout = 30
17 shutdown_timeout = 5
18 folder = os.getcwd()
19 extcron = None
20 nocron = None
This file contains the web2py defaults. If you edit this file, you need to
import it explicitly with the -L command-line option. It only works if you
run web2py as a Windows Service.
the core 137
The web2py workflow is the following:
An HTTP requests arrives to the web server (the built-in Rocket server or
a different server connected to web2py via WSGI or another adapter). The
web server handles each request in its own thread, in parallel.
The HTTP request header is parsed and passed to the dispatcher
(explained later in this chapter).
The dispatcher decides which of the installed application will handle the
request and maps the PATH_INFO in the URL into a function call. Each
URL corresponds to one function call.
Requests for files in the static folder are handled directly, and large files
are automatically streamed to the client.
Requests for anything but a static file are mapped into an action (i.e. a
function in a controller file, in the requested application).
Before calling the action, a few things happen: if the request header
contains a session cookie for the app, the session object is retrieved; if
not, a session id is created (but the session file is not saved until later); an
execution environment for the request is created; models are executed in
this environment.
Finally the controller action is executed in the pre-built environment.
If the action returns a string, this is returned to the client (or if the action
returns a web2py HTML helper object, it is serialized and returned to the
If the action returns an iterable, this is used to loop and stream the data to
the client.
• If the action returns a dictionary, web2py tries to locate a view to
render the dictionary. The view must have the same name as the action
(unless specified otherwise) and the same extension as the requested page
(defaults to.html); on failure, web2py may pick up a generic view (if
138 web2py full-stack web framework,4th edition
available and if enabled). The view sees every variable defined in the
models as well as those in the dictionary returned by the action, but does
not see global variables defined in the controller.
The entire user code is executed in a single transaction unless specified
If the user code succeeds, the transaction is committed.
If the user code fails, the traceback is stored in a ticket, and a ticket ID is
issued to the client. Only the system administrator can search and read
the tracebacks in tickets.
There are some caveats to keep in mind:
Models in the same folder/subfolder are executed in alphabetical order.
Any variable defined in a model will be visible to other models following
alphabetically, to the controllers, and to the views.
Models in subfolders are executed conditionally. For example, if the user
has requested "/a/c/f" where "a" is the application, "c" is the controller,
and "f" is the function (action), then the following models are executed:
The requested controller is executed and the requested function is called.
This means all top-level code in the controller is also executed at every
request for that controller.
The view is only called if the action returns a dictionary.
If a view is not found, web2py tries to use a generic view. By default,
generic views are disabled, although the ’welcome’ app includes a line in
/models/db.py to enable them on localhost only. They can be enabled
per extension type and per action (using response.generic_patterns). In
general, generic views are a development tool and typically should not be
used in production. If you want some actions to use a generic view, list
those actions in response.generic_patterns (discussed in more detail in the
chapter on Services).
the core 139
The possible behaviors of an action are the following:
Return a string
1def index(): return 'data'
Return a dictionary for a view:
1def index(): return dict(key='value')
Return all local variables:
1def index(): return locals()
Redirect the user to another page:
1def index(): redirect(URL('other_action'))
Return an HTTP page other than "200 OK":
1def index(): raise HTTP(404)
Return a helper (for example, a FORM):
1def index(): return FORM(INPUT(_name='test'))
(this is mostly used for Ajax callbacks and components, see chapter 12)
When an action returns a dictionary, it may contain code generated by
helpers, including forms based on database tables or forms from a factory,
for example:
1def index(): return dict(form=SQLFORM.factory(Field('name')).process())
(all forms generated by web2py use postbacks, see chapter 3)
web2py maps a URL of the form:
to the function f() in controller "c.py" in application "a". If fis not present,
web2py defaults to the index controller function. If cis not present, web2py
140 web2py full-stack web framework,4th edition
defaults to the "default.py" controller, and if ais not present, web2py defaults
to the init application. If there is no init application, web2py tries to run the
welcome application. This is shown schematically in the image below:
(The names of the default application, controller and function can be
overridden in routes.py; see Default Application, Controller and Function below.
By default, any new request also creates a new session. In addition, a session
cookie is returned to the client browser to keep track of the session.
The extension .html is optional; .html is assumed as default. The extension
determines the extension of the view that renders the output of the controller
function f(). It allows the same content to be served in multiple formats
(html, xml, json, rss, etc.).
Functions that take arguments or start with a double underscore are not
publicly exposed and can only be called by other functions.
There is an exception made for URLs of the form:
There is no controller called "static". web2py interprets this as a request for
the file called "filename" in the subfolder "static" of the application "a".
When static files are downloaded, web2py does not create a session, nor
does it issue a cookie or execute the models. web2py always streams static
the core 141
files in chunks of 1MB, and sends PARTIAL CONTENT when the client
sends a RANGE request for a subset of the file. web2py also supports the
IF_MODIFIED_SINCE protocol, and does not send the file if it is already
stored in the browser’s cache and if the file has not changed since that
When linking to an audio or video file in the static folder, if you want to
force the browser to download the file instead of streaming the audio/video
via a media player, add ?attachment to the URL. This tells web2py to set
the Content-Disposition header of the HTTP response to "attachment". For
1<a href="/app/static/my_audio_file.mp3?attachment">Download</a>
When the above link is clicked, the browser will prompt the user to download
the MP3file rather than immediately streaming the audio. (As discussed
below, you can also set HTTP response headers directly by assigning a dict of
header names and their values to response.headers.)
web2py maps GET/POST requests of the form:
to function fin controller "c.py" in application a, and it stores the URL
parameters in the request variable as follows:
1request.args = ['x','y','z']
1request.vars = {'p':1, 'q':2}
1request.application = 'a'
2request.controller = 'c'
3request.function = 'f'
In the above example, both request.args[i] and request.args(i) can be used
to retrieve the i-th element of the request.args, but while the former raises an
exception if the list does not have such an index, the latter returns None in
this case.
142 web2py full-stack web framework,4th edition
stores the full URL of the current request (not including GET variables).
defaults False but it is True if web2py determines that the action was called
by an Ajax request.
If the request is an Ajax request and it is initiated by a web2py component,
the name of the component can be found in:
Components are discussed in more detail in Chapter 12.
If the HTTP request is a GET, then request.env.request_method is set to "GET";
if it is a POST, request.env.request_method is set to "POST". URL query
variables are stored in the request.vars Storage dictionary; they are also
stored in request.get_vars (following a GET request) or request.post_vars
(following a POST request). web2py stores WSGI and web2py environment
variables in request.env, for example:
1request.env.path_info = 'a/c/f'
and HTTP headers into environment variables, for example:
1request.env.http_host = ''
Notice that web2py validates all URLs to prevent directory traversal attacks.
URLs are only allowed to contain alphanumeric characters, underscores, and
slashes; the args may contain non-consecutive dots. Spaces are replaced by
underscores before validation. If the URL syntax is invalid, web2py returns
an HTTP 400 error message [47,48].
If the URL corresponds to a request for a static file, web2py simply reads and
returns (streams) the requested file.
If the URL does not request a static file, web2py processes the request in the
following order:
Parses cookies.
the core 143
Creates an environment in which to execute the function.
Initializes request,response,cache.
Opens the existing session or creates a new one.
Executes the models belonging to the requested application.
Executes the requested controller action function.
If the function returns a dictionary, executes the associated view.
On success, commits all open transactions.
Saves the session.
Returns an HTTP response.
Notice that the controller and the view are executed in different copies of
the same environment; therefore, the view does not see the controller, but
it sees the models and it sees the variables returned by the controller action
If an exception (other than HTTP) is raised, web2py does the following:
Stores the traceback in an error file and assigns a ticket number to it.
Rolls back all open transactions.
Returns an error page reporting the ticket number.
If the exception is an HTTP exception, this is assumed to be the intended
behavior (for example, an HTTP redirect), and all open database transactions
are committed. The behavior after that is specified by the HTTP exception
itself. The HTTP exception class is not a standard Python exception; it is
defined by web2py.
The web2py libraries are exposed to the user applications as global objects.
For example (request,response,session,cache), classes (helpers, validators,
DAL API), and functions (Tand redirect).
144 web2py full-stack web framework,4th edition
These objects are defined in the following core files:
2gluon/__init__.py gluon/highlight.py gluon/restricted.py gluon/streamer.py
3gluon/admin.py gluon/html.py gluon/rewrite.py gluon/template.py
4gluon/cache.py gluon/http.py gluon/rocket.py gluon/storage.py
5gluon/cfs.py gluon/import_all.py gluon/sanitizer.py gluon/tools.py
6gluon/compileapp.py gluon/languages.py gluon/serializers.py gluon/utils.py
7gluon/contenttype.py gluon/main.py gluon/settings.py gluon/validators.py
8gluon/dal.py gluon/myregex.py gluon/shell.py gluon/widget.py
9gluon/decoder.py gluon/newcron.py gluon/sql.py gluon/winservice.py
10 gluon/fileutils.py gluon/portalocker.py gluon/sqlhtml.py gluon/xmlrpc.py
11 gluon/globals.py gluon/reserved_sql_keywords.py
The tar gzipped scaffolding app that ship with web2py is
It is created upon installation and overwritten on upgrade.
The first time you start web2py, two new folders are created: deposit and
applications. The "welcome" app is zipped into a "welcome.w2p" file to be
used as a scaffolding app. The first time you start web2py, two new folders
are created: deposit and applications. The "welcome" app is zipped into a
"welcome.w2p" file to be used as scaffolding app. The deposit folder is used as
temporary storage for installing and uninstalling applications.
web2py unit-tests are in
There are handlers for connecting with various web servers:
1cgihandler.py # discouraged
2gaehandler.py # for Google App Engine
3fcgihandler.py # for FastCGI
4wsgihandler.py # for WSGI
5isapiwsgihandler.py # for IIS
6modpythonhandler.py # deprecated
("fcgihandler" calls "gluon/contrib/gateways/fcgi.py" developed by Allan
Saddi) and
the core 145
which is a script to interface with many different web servers, described in
Chapter 13.
There are three example files:
The former is an optional configuration file that can be passed to web2py.py
with the -L option. The second is an example of a URL mapping file. It is
loaded automatically when renamed "routes.py". The third is an alternative
syntax for URL mapping, and can also be renamed (or copied to) "routes.py".
The files
are configuration files used for deployment on the Google App Engine. You
can read more about them in the Deployment Recipes chapter and on the
Google Documentation pages.
There are also additional libraries, usually developed by a third party:
feedparser [28] by Mark Pilgrim for reading RSS and Atom feeds:
markdown2[29] by Trent Mick for wiki markup:
markmin markup:
pyfpdf created my Mariano Reingart for generating PDF documents:
This is not documented in this book but it is hosted and documented here: