avatar

Derek Zeng

Loneliness is the gift of life

Python built-in tools that you must know

This is a study note about Python basics.

(based on python 3.6)


itertools

Helper functions returns iterator.

import itertools as it

Infinite sequence

count, cycle, repeat

c = it.count(10)
print(next(c)) # 10
print(next(c)) # 11
print(next(c)) # 12

increment_count = map(lambda co: co+1, c)
print(next(increment_count)) # 14
print(next(increment_count)) # 15
print(next(increment_count)) # 16

For cycle, it takes in as argument an iterable, and infinitely cycle through the iterable.

For repeat, it repeat something for a number of times or infinite times.

accumulate

Similar to reduce, but it produces a list of intermediate values

ls = [1,2,3,4,5,6]
print(it.accumulate(ls)) # 1,3,6,10,15,21
print(it.accumulate(ls, min)) # 1,1,1,1,1,1

chain

Concatenate iterable.

An example of using chain to shallowly flatten lists. (This is from official doc)

def flatten(listOfLists):
    "Flatten one level of nesting"
    return chain.from_iterable(listOfLists)

However I found another way to implement deep flatten using yield from

def flatten(iterable):
    for i in iterable:
        try:
            iter(i)
            yield from flatten(i)
        except:
            yield i

def fibo(n):
    a, b = 0, 1
    while n > 0:
        a, b = b, a+b
        n -= 1
        yield b

flatten([1, [fibo(5)], [3, [4]]]) # [1, 1, 2, 3, 5, 8, 3, 4]

combination, permutation

Self-explanatory

filterfalse

Opposite to filter

islice

Slice an iterable

takewhile, dropwhile

tee

Clone an iterable to several

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return zip(a, b)

zip_longest

Like zip, but when two lists are of unequal length, fill the missing with a default value


collections

abc

Abstract base class. Easier way to do duck typing

namedtuple

Like struct

deque

Double-ended queue, fast append and pop

Counter(dict)

A sorted dictionary with frequency count. It provides convenient methods like most_common and set operators

defaultdict, OrderedDict

OrderedDict is like TreeMap in Java. It can be used to implement LinkedHashMap

class LastUpdatedOrderedDict(OrderedDict):
    'Store items in the order the keys were last added'

    def __setitem__(self, key, value):
        if key in self:
            del self[key]
        OrderedDict.__setitem__(self, key, value)

defaultdict is just dict with a default value.

listdict = it.defaultdict(list)
listdict['a'].append(123) # {'a': [123]}
listdict['a'].append(345) # {'a': [123, 345]}

# an easy way to implement trie
Trie = lambda: defaultdict(Trie)

trie = Trie()
for w in words:
    reduce(dict.__getitem__, trie, w)

ChainMap

It's like the Object.assign method in es6 but with more power.

a = {'a': 123, 'b': 456}
b = {'a': 33, 'b': 23, 'c': 'hello'}

l = it.ChainMap(a, b)

print(l.get('b')) # 456

# only add/update/del the first one, 
l.update({'a': 11})
print(l.get('a')) # 11

l.pop('a') # 33
print(l.get('a'))

print(l.parents) # ChainMap({'a': 33, 'b': 23, 'c': 'hello'}); equal to [b]
print(l.parents[0] == b) # True

# new copy with empty dict prepended
new_l = l.new_child() # ChainMap({}, {'b': 456}, {'a': 33, 'b': 23, 'c': 'hello'})
print(new_l)

Built in functions

open, input, print

locals, globals, vars

Env inspection

abs, min, max, sum, pow, round, complex, divmod

Math functions

hex, bin, oct

Convert a decimal number to hex, binary, or oct representation

int, str, bool, float

Type casting

chr, ord

print(chr(26366)) # 曾
print(ord('曾')) # 26366

getattr, setattr, hasattr, delattr

Attributes functions

hash, id

Identity functions

# id is implicitly used to construct hash value of instance object to be used in dict

iter, next

Iterator functions

list, set, tuple

Convert to list, set or tuple, evaluating the generator if exists.

reversed, sorted, zip, enumerate, map, filter, any, all, len, frozenset, range

List operations

bytearray, bytes

Binary data structure. Mutable and immutable

callable, issubclass, isinstance, type

Type testing functions.

property, super, staticmethod, classmethod

Class decorators and super class accessor

help, dir

Get details of an object

repr, ascii

Return string representation of the object

eval, exec

Dynamic code execution


functools

wraps

Needed for properly implementing decorator https://coolshell.cn/articles/11265.html

reduce

Move from builtin function by Guido van Rossum because he thinks the explicit way is better.

partial

Similar to lodash partial.

lru_cache

Very good decorator to implement memoization.

cmp_to_key

In py3, there is no more cmp param for sort related functions. Instead, all such use case should be replaced by a key function. This is to follow python philosophy of one way doing things.

cmp_to_key is used to convert old-style cmp function to a key function.

singledispatch

Used for generic functions. i.e. invoke a method based on argument type.

(End of article)