A Python program is read by a parser. Input to the parser is a stream of tokens, generated by the lexical analyzer(also known as the tokenizer). This chapter describes how the lexical analyzer brea...