Writing faster code

Writing faster code and not hating your job as a software developer

Writing faster Python

@SebaWitowski

Sebastian Witowski

# https://en.wikipedia.org/wiki/R_(programming_language)
install.packages("caTools")
library(caTools)
jet.colors <- colorRampPalette(c("green", "blue", "red",
                                 "cyan", "#7FFF7F",
                                 "yellow", "#FF7F00",
                                 "red", "#7F0000"))
m <- 1000
C <- complex( real=rep(seq(-2.10,0.8, length.out=m), each=m ),
              imag=rep(seq(-1.2,1.2, length.out=m), m ) )
C <- matrix(C,m,m)
Z <- 0
X <- array(0, c(m,m,20))
for (k in 1:20) {
  Z <- Z^2+C
  X[,,k] <- exp(-abs(Z))
}
write.gif(X, "Mandelbrot.gif", col=jet.colors, delay=900)
                            

Small scripts...

# Fibonacci sequence (from rosettacode)

recfibo <- function(n) {
  if ( n < 2 ) n
  else Recall(n-1) + Recall(n-2)
}

# Print the first 21 elements
print.table(lapply(0:20, recfibo))
                        

...will grow

Python was not made to be fast...

...but to make developers fast.

Would you like your FIRST program EVER to be like:

public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, world!");
    }
}
                        

or

print("Hello, world!")
                        

OPTIMIZATION

Rules of optimization:

1. Don't

2. Don't... yet

• Finish your code
• Have tests
• Now

3. Profile

Optimization is all about the speed

cProfile

pstats

RunSnakeRun

SnakeViz

SnakeViz

memory_profiler

$ pip install -U memory_profiler
                        

@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a
                        

memory_profiler

memory_profiler

mprof run example.py
mprof plot
                        

Levels of optimization

• Design
• Algorithms and data structures

sum = 0
for x in range(1, N + 1):
    sum += x
print sum
                        

print N * (1 + N) / 2
                            

Levels of optimization

• Design
• Algorithms and data structures
• Source code
• Build level
• Compile level
• Runtime level

Writing fast Python

a.k.a source code optimization

Setup

Python 3.5.1 (IPython 1.2.1)

def ultimate_answer_to_life():
    return 42
                        

>>> %timeit ultimate_answer_to_life()
10000000 loops, best of 3: 87.1 ns per loop
                        

2.72 × 10²¹ times faster than in The Hitchhiker's Guide to the Galaxy ;-)

#1 Count elements in a list

how_many = 0
for element in ONE_MILLION_ELEMENTS:
    how_many += 1
print how_many
                        

26.5 ms

print len(ONE_MILLION_ELEMENTS)
                        

96.7 ns

274 000 times faster

#4 Permissions or forgiveness ?

class Foo(object):
    hello = 'world'
foo = Foo()
                        

if hasattr(foo, 'hello'):
    foo.hello
                        

149 ns

try:
    foo.hello
except AttributeError:
    pass
                        

43.1 ns
3.5 times faster

#4 Permissions or forgiveness ?

if (hasattr(foo, 'foo') and hasattr(foo, 'bar')
    and hasattr(foo, 'baz')):
    foo.foo
    foo.bar
    foo.baz
                        

401 ns

try:
    foo.foo
    foo.bar
    foo.baz
except AttributeError:
    pass
                        

110 ns
3.64 times faster

#4 Permissions or forgiveness ?

class Bar(object):
    pass
bar = Bar()
                        

if hasattr(bar, 'hello'):
    bar.hello
                        

428 ns

try:
    bar.hello
except AttributeError:
    pass
                        

536 ns
25% slower

#5 Membership testing

def check_number(number):
    for item in MILLION_NUMBERS:
        if item == number:
            return True
    return False
                        

%timeit check_number(500000)
                        

18 ms

500000 in MILLION_NUMBERS
                            

#5 Membership testing

100 in MILLION_NUMBERS
                        

1.55 µs

999999 in MILLION_NUMBERS
                        

15.7 ms

#5 Membership testing

MILLION_SET = set(MILLION_NUMBERS)
%timeit 100 in MILLION_SET
                        

46.3 ns
33 times faster (vs list)

%timeit 999999 in MILLION_SET
                        

63.3 ns
248 000 times faster (vs list)

%timeit set(MILLION_NUMBERS)
                            

#9 Checking for True

if variable == True:
                        

35.8 ns

if variable is True:
                        

28.7 ns
24% faster

if variable:
                        

20.6 ns
73% faster

#9.1 Checking for False

if variable == False:
                        

35.1 ns

if variable is False:
                        

26.9 ns
30% faster

if not variable:
                        

19.8 ns
77% faster

#9.2 Checking for empty list

if len(a_list) == 0:
                        

91.7 ns

if a_list == []:
                        

56.3 ns
60% faster

if not a_list:
                        

32.4 ns
280% faster

#10 Def vs lambda

def greet(name):
    return 'Hello {}!'.format(name)
                        

329 ns

greet = lambda name: 'Hello {}!'.format(name)
                        

332 ns

#10 Def vs lambda

>>> dis.dis(greet)
 0 LOAD_CONST    1 ('Hello {}!')
 3 LOAD_ATTR     0 (format)
 6 LOAD_FAST     0 (name)
 9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 RETURN_VALUE
                        

#11 list() or []

list()
                        

104 ns

[]
                        

22.5 ns

4.6 times faster

#11.1 dict() or {}

dict()
                        

161 ns

{}
                        

93 ns

1.7 times faster