wiki:Development/Efficiency

Efficiency of Python Idioms

Profiling and improving the performance of any Python code requires some familiarity with the efficiency of key idioms in the Python language. As with most environments, there are multiple ways to accomplish the same task. Here we document some common Python idioms and our preferred form (based on measured performance).

Testing Truth

When constructing logical control expressions, how should we check for "truth"?

  • Guidance:
    • prefer
      if x:
         pass
      # or
      if not x:
         pass
      
    • avoid
      if x == True:
         pass
      # and
      if x is True:
         pass
      
  • Justification: source Download, results Download

Function Signatures

When defining and calling functions, Python allows several flexible formats, including classic positional arguments and keyword arguments. Arguments may provide default values. Variable (unspecified) positional and keyword arguments are allowed.

  • Guidance:
    • prefer explicit positional arguments with default values
    • avoid unspecified keyword arguments (e.g., the "**kwds" idiom)
  • Justification: source Download, results Download

Attributes, Methods, and Properties

When managing access to object attributes, Python allows direct access of the attribute, access through user-defined methods, or access through "@property" decorators.

  • Guidance:
    • Methods are roughly 3x direct attribute access; Properties are nearly 4x direct attribute access.
    • Be aware of the performance differences, but allow good API design and programming practice to dictate the access method.
  • Justification: source Download, results Download

Accessing Dictionaries

Working with dictionaries can become tricky when the code depends on if the specified key is already in the dictionary. The following guidance should hold for types that do not implement a custom implementation of "__hash__()".

  • Guidance:
    • Retrieving a value from a dict that is guaranteed to be present
      • prefer
        d[x]
        
      • avoid
        d.get(x)
        
    • Retrieving a value from a dict if present, otherwise return a fixed value without updating the dictionary
      • prefer
        d[x] if x in d else None
        
      • avoid
        d.get(x, None)
        
    • Setting a value in a dict if the key is not present
      • prefer
        if x not in d:
           d[x] = v
        
      • avoid
        d.setdefault(x, v)
        
    • Returning a value from d, setting a default if the key is not present
      • prefer
        if x in d:
           a = d[x]
        else:
           a = v
           d[x] = a
        
      • avoid
        a = d.setdefault(x, v)
        
    • (Iteratively) constructing a dictionary
      • prefer passing a tuple-generator directly to the dict constructor, e.g.:
        ans = dict( d )
        # or
        ans = dict( d.iteritems() )
        
      • fallback to setting the elements one at a time, e.g.:
        ans = {}
        for i,v in d.iteritems():
           ans[i] = v
        
      • avoid using a generator that constructs and discards an intermediate tuple, e.g.:
        ans = dict((i,v) for i,v in d.iteritems())
        
  • Justification: source Download, results Download

Attachments