A Python Book - Dave's page

Contents
1 Part 1 Beginning Python...........................................................................................10
1.1 Introductions Etc...................................................................................................10
1.1.1 Resources.......................................................................................................11
1.1.2 A general description of Python....................................................................12
1.1.3 Interactive Python..........................................................................................15
1.2 Lexical matters......................................................................................................15
1.2.1 Lines..............................................................................................................15
1.2.2 Comments......................................................................................................16
1.2.3 Names and tokens..........................................................................................16
1.2.4 Blocks and indentation..................................................................................16
1.2.5 Doc strings.....................................................................................................17
1.2.6 Program structure..........................................................................................17
1.2.7 Operators.......................................................................................................18
1.2.8 Also see.........................................................................................................19
1.2.9 Code evaluation.............................................................................................19
1.3 Statements and inspection preliminaries...........................................................20
1.4 Builtin datatypes.................................................................................................21
1.4.1 Numeric types................................................................................................21
1.4.2 Tuples and lists..............................................................................................21
1.4.3 Strings............................................................................................................24
1.4.3.1 The new string.format method...............................................................26
1.4.3.2 Unicode strings......................................................................................27
1.4.4 Dictionaries....................................................................................................29
1.4.5 Files...............................................................................................................32
1.4.6 Other builtin types........................................................................................35
1.4.6.1 The None value/type..............................................................................35
1.4.6.2 Boolean values.......................................................................................36
1.4.6.3 Sets and frozensets.................................................................................36
1.5 Functions and Classes A Preview......................................................................36
1.6 Statements.............................................................................................................37
1.6.1 Assignment statement....................................................................................37
1.6.2 import statement............................................................................................39
1.6.3 print statement...............................................................................................41
1.6.4 if: elif: else: statement...................................................................................43
1.6.5 for: statement.................................................................................................44
1.6.6 while: statement.............................................................................................48
Page 3

A Python Book
1.6.7 continue and break statements.......................................................................48
1.6.8 try: except: statement.....................................................................................49
1.6.9 raise statement...............................................................................................51
1.6.10 with: statement.............................................................................................52
1.6.10.1 Writing a context manager...................................................................52
1.6.10.2 Using the with: statement....................................................................53
1.6.11 del................................................................................................................54
1.6.12 case statement..............................................................................................55
1.7 Functions, Modules, Packages, and Debugging....................................................55
1.7.1 Functions.......................................................................................................55
1.7.1.1 The def statement...................................................................................55
1.7.1.2 Returning values....................................................................................55
1.7.1.3 Parameters..............................................................................................56
1.7.1.4 Arguments..............................................................................................56
1.7.1.5 Local variables.......................................................................................57
1.7.1.6 Other things to know about functions....................................................57
1.7.1.7 Global variables and the global statement.............................................58
1.7.1.8 Doc strings for functions.......................................................................60
1.7.1.9 Decorators for functions........................................................................60
1.7.2 lambda...........................................................................................................61
1.7.3 Iterators and generators.................................................................................62
1.7.4 Modules.........................................................................................................67
1.7.4.1 Doc strings for modules.........................................................................68
1.7.5 Packages........................................................................................................68
1.8 Classes...................................................................................................................69
1.8.1 A simple class................................................................................................69
1.8.2 Defining methods..........................................................................................70
1.8.3 The constructor..............................................................................................70
1.8.4 Member variables..........................................................................................70
1.8.5 Calling methods.............................................................................................71
1.8.6 Adding inheritance........................................................................................71
1.8.7 Class variables...............................................................................................72
1.8.8 Class methods and static methods.................................................................72
1.8.9 Properties.......................................................................................................74
1.8.10 Interfaces.....................................................................................................75
1.8.11 Newstyle classes.........................................................................................75
1.8.12 Doc strings for classes.................................................................................77
1.8.13 Private members..........................................................................................77
1.9 Special Tasks.........................................................................................................77
1.9.1 Debugging tools.............................................................................................77
Page 4

A Python Book
1.9.2 File input and output......................................................................................78
1.9.3 Unit tests........................................................................................................80
1.9.3.1 A simple example..................................................................................80
1.9.3.2 Unit test suites........................................................................................81
1.9.3.3 Additional unittest features....................................................................83
1.9.3.4 Guidance on Unit Testing......................................................................85
1.9.4 doctest............................................................................................................85
1.9.5 The Python database API..............................................................................87
1.9.6 Installing Python packages............................................................................88
1.10 More Python Features and Exercises..................................................................89
2 Part 2 Advanced Python............................................................................................90
2.1 Introduction Python 201 (Slightly) Advanced Python Topics.......................90
2.2 Regular Expressions..............................................................................................90
2.2.1 Defining regular expressions.........................................................................90
2.2.2 Compiling regular expressions......................................................................91
2.2.3 Using regular expressions..............................................................................91
2.2.4 Using match objects to extract a value..........................................................92
2.2.5 Extracting multiple items..............................................................................93
2.2.6 Replacing multiple items...............................................................................94
2.3 Iterator Objects......................................................................................................96
2.3.1 Example A generator function....................................................................98
2.3.2 Example A class containing a generator method......................................100
2.3.3 Example An iterator class.........................................................................102
2.3.4 Example An iterator class that uses yield.................................................104
2.3.5 Example A list comprehension.................................................................105
2.3.6 Example A generator expression..............................................................105
2.4 Unit Tests............................................................................................................106
2.4.1 Defining unit tests........................................................................................106
2.4.1.1 Create a test class.................................................................................106
2.5 Extending and embedding Python......................................................................109
2.5.1 Introduction and concepts............................................................................109
2.5.2 Extension modules.......................................................................................110
2.5.3 SWIG...........................................................................................................112
2.5.4 Pyrex............................................................................................................115
2.5.5 SWIG vs. Pyrex...........................................................................................120
2.5.6 Cython.........................................................................................................120
2.5.7 Extension types............................................................................................122
2.5.8 Extension classes.........................................................................................122
2.6 Parsing.................................................................................................................122
2.6.1 Special purpose parsers...............................................................................123
Page 5

A Python Book
2.6.2 Writing a recursive descent parser by hand.................................................124
2.6.3 Creating a lexer/tokenizer with Plex...........................................................131
2.6.4 A survey of existing tools............................................................................141
2.6.5 Creating a parser with PLY.........................................................................141
2.6.6 Creating a parser with pyparsing.................................................................148
2.6.6.1 Parsing commadelimited lines............................................................148
2.6.6.2 Parsing functors...................................................................................149
2.6.6.3 Parsing names, phone numbers, etc.....................................................150
2.6.6.4 A more complex example....................................................................151
2.7 GUI Applications................................................................................................153
2.7.1 Introduction.................................................................................................153
2.7.2 PyGtk...........................................................................................................153
2.7.2.1 A simple message dialog box..............................................................153
2.7.2.2 A simple text input dialog box.............................................................156
2.7.2.3 A file selection dialog box...................................................................158
2.7.3 EasyGUI......................................................................................................160
2.7.3.1 A simple EasyGUI example................................................................161
2.7.3.2 An EasyGUI file open dialog example................................................161
2.8 Guidance on Packages and Modules...................................................................161
2.8.1 Introduction.................................................................................................161
2.8.2 Implementing Packages...............................................................................162
2.8.3 Using Packages............................................................................................162
2.8.4 Distributing and Installing Packages...........................................................162
2.9 End Matter...........................................................................................................164
2.9.1 Acknowledgements and Thanks..................................................................164
2.9.2 See Also.......................................................................................................164
3 Part 3 Python Workbook.........................................................................................165
3.1 Introduction.........................................................................................................165
3.2 Lexical Structures................................................................................................165
3.2.1 Variables and names....................................................................................165
3.2.2 Line structure...............................................................................................167
3.2.3 Indentation and program structure...............................................................168
3.3 Execution Model.................................................................................................169
3.4 Builtin Data Types.............................................................................................170
3.4.1 Numbers......................................................................................................170
3.4.1.1 Literal representations of numbers......................................................171
3.4.1.2 Operators for numbers.........................................................................173
3.4.1.3 Methods on numbers............................................................................175
3.4.2 Lists.............................................................................................................175
3.4.2.1 Literal representation of lists...............................................................176
Page 6

A Python Book
3.4.2.2 Operators on lists.................................................................................178
3.4.2.3 Methods on lists...................................................................................178
3.4.2.4 List comprehensions............................................................................180
3.4.3 Strings..........................................................................................................182
3.4.3.1 Characters............................................................................................183
3.4.3.2 Operators on strings.............................................................................184
3.4.3.3 Methods on strings...............................................................................185
3.4.3.4 Raw strings..........................................................................................187
3.4.3.5 Unicode strings....................................................................................188
3.4.4 Dictionaries..................................................................................................190
3.4.4.1 Literal representation of dictionaries...................................................190
3.4.4.2 Operators on dictionaries.....................................................................191
3.4.4.3 Methods on dictionaries.......................................................................192
3.4.5 Files.............................................................................................................195
3.4.6 A few miscellaneous data types..................................................................197
3.4.6.1 None.....................................................................................................197
3.4.6.2 The booleans True and False...............................................................197
3.5 Statements...........................................................................................................198
3.5.1 Assignment statement..................................................................................198
3.5.2 print statement.............................................................................................200
3.5.3 if: statement exercises..................................................................................201
3.5.4 for: statement exercises...............................................................................202
3.5.5 while: statement exercises...........................................................................205
3.5.6 break and continue statements.....................................................................206
3.5.7 Exceptions and the try:except: and raise statements...................................207
3.6 Functions.............................................................................................................210
3.6.1 Optional arguments and default values.......................................................211
3.6.2 Passing functions as arguments...................................................................213
3.6.3 Extra args and keyword args.......................................................................214
3.6.3.1 Order of arguments (positional, extra, and keyword args)..................216
3.6.4 Functions and ducktyping and polymorphism...........................................216
3.6.5 Recursive functions.....................................................................................217
3.6.6 Generators and iterators...............................................................................219
3.7 Objectoriented programming and classes..........................................................223
3.7.1 The constructor............................................................................................224
3.7.2 Inheritance Implementing a subclass.......................................................225
3.7.3 Classes and polymorphism..........................................................................227
3.7.4 Recursive calls to methods..........................................................................228
3.7.5 Class variables, class methods, and static methods.....................................230
3.7.5.1 Decorators for classmethod and staticmethod.....................................233
Page 7

A Python Book
3.8 Additional and Advanced Topics........................................................................234
3.8.1 Decorators and how to implement them......................................................234
3.8.1.1 Decorators with arguments..................................................................235
3.8.1.2 Stacked decorators...............................................................................236
3.8.1.3 More help with decorators...................................................................238
3.8.2 Iterables.......................................................................................................239
3.8.2.1 A few preliminaries on Iterables..........................................................239
3.8.2.2 More help with iterables......................................................................240
3.9 Applications and Recipes....................................................................................240
3.9.1 XML SAX, minidom, ElementTree, Lxml..............................................241
3.9.2 Relational database access...........................................................................249
3.9.3 CSV comma separated value files...........................................................255
3.9.4 YAML and PyYAML..................................................................................256
3.9.5 Json..............................................................................................................258
4 Part 4 Generating Python Bindings for XML.........................................................260
4.1 Introduction.........................................................................................................260
4.2 Generating the code.............................................................................................261
4.3 Using the generated code to parse and export an XML document.....................263
4.4 Some command line options you might want to know.......................................263
4.5 The graphical frontend.......................................................................................264
4.6 Adding applicationspecific behavior.................................................................265
4.6.1 Implementing custom subclasses................................................................265
4.6.2 Using the generated "API" from your application......................................266
4.6.3 A combined approach..................................................................................267
4.7 Special situations and uses..................................................................................269
4.7.1 Generic, typeindependent processing.........................................................269
4.7.1.1 Step 1 generate the bindings............................................................270
4.7.1.2 Step 2 add applicationspecific code................................................270
4.7.1.3 Step 3 write a test/driver harness.....................................................274
4.7.1.4 Step 4 run the test application..........................................................276
4.8 Some hints...........................................................................................................276
4.8.1 Children defined with maxOccurs greater than 1........................................276
4.8.2 Children defined with simple numeric types...............................................277
4.8.3 The type of an element's character content..................................................277
4.8.4 Constructors and their default values..........................................................277