Python Primer

Python is extremely useful in almost every area of software development. In embedded engineering Python is most useful for CI/CD automation. In 2023, according to PYPL (PopularitY of Programming Language) python was #1 programming language claiming 28% market share.

I have put together this cheatsheet to help you go from zero to hero in python in the shortest amount of time possible. We will cover a wide range of topics going from the simplest variable declarations, to abstract classes and more complex design patterns.

This cheatsheet is designed to easy to scroll on a single page so that you can quickly find the information you need.

Variables and Data Types

Python is a dynamically typed programming language. You do not need to declare variable types, rather the type of the variable is determined by the value that it contains.

name = "Foo"    # String
age = 30        # Integer
a_number = 5.9  # Float
a_flag = True   # Boolean

Lists

Python is strong with lists. Lists in python are ordered collections that can hold items of different types.

items = ["cat", "dog", "tree"]
items.append("car")  # Add an item
print(items[1])      # Access the second item

Dictionaries

Dictionaries provide the basic means of structured data representation in python. Dictionaries store key-value pairs.

person = {"name": "John", "age": 30}
person["height"] = 5.9  # Add a new key-value pair
print(person["name"])   # Access value by key

Deques

from collections import deque
 
queue = deque(["Eric", "John", "Michael"])
queue.append("Terry")           # Terry arrives
queue.popleft()                 # The first to arrive now leaves
 
print(queue)                    # Remaining queue in order of arrival

Enums

from enum import Enum, auto
 
class Color(Enum):
    RED = auto()
    GREEN = auto()
    BLUE = auto()
 
print(Color.RED)

Data Classes

Data classes give you a simple way to create structured objects that contain data. They are easier to access than dict types since you can use the dot . notation.

from dataclasses import dataclass
 
@dataclass
class Product:
    name: str
    price: float
 
product = Product("Widget", 19.99)
print(product.name)

Control Structures

If Statements

if age > 18:
    print("Adult")
else:
    print("Not Adult")

You can also write it like this:

variable = "Adult" if age > 18 else "Not Adult"

For Loops

for item in item_list:
    print(fruit)

Range loops:

for i in range(0, 10):
    print(i)

You can also just loop without using the value. In that case use underscore _ to denote placeholder variables:

for _ in range(0, 10):
    pass

While Loops

while len(items):
    items.pop()  # pops last item and removes it from the list

Functions

def length(name: str) -> int:
    return len(str)
print(length("foo"))

You can use type hints as I did above in order to be able to use static analysis tools like mypy to help you find bugs in your program. In general you should always use type hints with all variables.

Classes

Object-oriented programming is easy in python.

class Person:
    def __init__(self, name: str, age: int):
        self._name = name
        self._age = age
 
    def greet(self):
        print(f"Hello, my name is {self._name} and I am {self._age} years old.")
 
person = Person("John", 30)
person.greet()

Use underscore _ prefix to designate "private" properties which should not be accessed directly. Python will allow you to access these properties anyway but static analysis checks will warn you about such unintended access.

Abstract classes

from abc import ABC, abstractmethod
 
class AbstractClass(ABC):
    @abstractmethod
    def my_method(self):
        pass
 
class ConcreteClass(AbstractClass):
    def my_method(self):
        print("Implementing the abstract method")
 
obj = ConcreteClass()
obj.my_method()

Meta classes

Metaclasses are the 'classes of classes', allowing you to control the creation of classes.

class Meta(type):
    def __new__(cls, name, bases, dct):
        # Custom processing before class creation
        return super().__new__(cls, name, bases, dct)
 
class MyClass(metaclass=Meta):
    pass

Dynamic attributes

You can use __getattr__, __setattr__, and @property for dynamic attribute management.

class MyClass:
    def __init__(self, value):
        self._value = value
 
    def __getattr__(self, name):
        # Custom behavior for undefined attributes
        if name == "dynamic_attribute":
            return "Dynamic Value"
        raise AttributeError(f"{name} not found")
 
    @property
    def value(self):
        return self._value
 
    @value.setter
    def value(self, value):
        self._value = value
 
obj = MyClass(10)
print(obj.dynamic_attribute)  # Dynamic Value
obj.value = 20
print(obj.value)

List Comprehensions

List comprehensions provide a concise way to process lists. They allow you to easily loop through items and modify them returning a new list as result. All in one line of Python code.

squares = [x**2 for x in range(10)]
print(squares)

Dictionary Comprehensions

Similar to list comprehensions, but for dictionaries.

Example:

square_dict = {x: x**2 for x in range(5)}
print(square_dict)
# result:
{0: 0, 1: 1, 2: 4, 3: 9, 4: 16}

Lambda Functions

You can define anonymous functions and pass them as variables using lambda keyword.

Example:

times_two = lambda x: x * 2
print(times_two(5))

Map and Filter

map and filter are functions that apply a function to each item in a list. You can use lambda functions and pass them directly as parameter to map or filter.

nums = [1, 2, 3, 4]
squared = list(map(lambda x: x**2, nums))
print(squared)

even_nums = list(filter(lambda x: x % 2 == 0, nums))
print(even_nums)

You can also filter like this:

[x for x in [1,2,3,4] if x < 3]
# result:
[1, 2]

Exception Handling

Use try and except blocks to handle exceptions.

try:
    result = 42 / 0
except ZeroDivisionError:
    print("Division by zero!")

Pass exception further:

try:
    raise ValueError("Initial error")
except ValueError as e:
    raise RuntimeError("Handling the error") from e

YAML

Yaml is extremely powerful text based structured data description language that supports data descriptions from basic objects to complex hierarchies with inheritance.

import yaml
document = """
  a: 1
  b:
    c: 3
    d: 4
"""
print yaml.dump(yaml.load(document))

JSON

JSON is also very easy to parse in python. Although not as flexible and powerful as yaml.

import json
 
# Converting from JSON
user_json = '{"name": "John", "age": 30}'
user = json.loads(user_json)
print(user['name'])
 
# Converting to JSON
user_dict = {"name": "John", "age": 30}
user_json = json.dumps(user_dict)
print(user_json)

Virtual Environments

Starting from recent 2023 ubuntu release, all python package installation should be done through virtual environments.

Creating a virtual environment:

python3 -m venv my_venv

Activating a virtual environment:

source my_venv/bin/activate

You can create a default virtual environment in your home directory and then activate it by default in every terminal by placing the activation code above into your .bashrc.

Modules and Packages

Any python file is treated as a module, meaning you can import classes and functions from it using the import statement.

import math
from .my_local_module import func
print(math.sqrt(16))
func()

A package in python is a collection of modules. It is defined by creating a special __init__.py file in the same directory as your modules. In this file you can then import modules that you expect to also be exported from your package.

Decorators

Decorators in python allow you to enhance your functions without modifying their code.

def my_decorator(func):
    def wrapper():
        print("This is run before the function is called.")
        func()
        print("This is run after the function is called.")
    return wrapper
 
@my_decorator
def say_hello():
    print("Hello!")
 
say_hello()

Decorators can also return a dynamic functions which allows you to add additional logic in your decorators and even instantiate the same decorator multiple times:

def repeat(times):
    def decorator_repeat(func):
        def wrapper(*args, **kwargs):
            for _ in range(times):
                result = func(*args, **kwargs)
            return result
        return wrapper
    return decorator_repeat
 
@repeat(times=3)
def greet(name):
    print(f"Hello {name}")
 
greet("Alice")

Decorators with arguments:

def decorator_with_args(arg1, arg2):
    def decorator(func):
        def wrapper(*args, **kwargs):
            print(f"Arguments passed to decorator: {arg1}, {arg2}")
            return func(*args, **kwargs)
        return wrapper
    return decorator
 
@decorator_with_args('hello', 'world')
def my_function():
    print("Function execution")
 
my_function()

Generators

Generators allow you to create iterators in a more straightforward and memory-efficient way.

def countdown(num):
    while num > 0:
        yield num
        num -= 1
 
for i in countdown(5):
    print(i)

Thread Concurrency

from threading import Thread
 
def print_numbers():
    for i in range(5):
        print(i)
 
def print_letters():
    for letter in ['a', 'b', 'c', 'd', 'e']:
        print(letter)
 
thread1 = Thread(target=print_numbers)
thread2 = Thread(target=print_letters)
 
thread1.start()
thread2.start()
 
thread1.join()
thread2.join()

Process Concurrency

Forking processes in python is easy.

from multiprocessing import Process
 
def task():
    print("This is a multiprocessing task.")
 
process = Process(target=task)
process.start()
process.join()

Regular expressions

Regular expressions is way of describing patterns that can directly parse a class of input data called "regular languages" in automata theory.

In python you can use these expressions to match complex patterns in strings.

import re
 
pattern = re.compile(r'\b[A-Za-z]+\b')
sentence = "Regex is fun!"
matches = pattern.findall(sentence)
print(matches)

Asynchronous IO

In contrast to threads which implement preemptive concurrency (through operating system functions), asynchronous IO interfaces with kpoll, select and epoll family of calls internally. This allows your code to "sleep" while it is waiting for asynchronous events being delivered from multiple sources. This is why async always involves a main loop that sleeps most of the time and only wakes up and runs your code when events occur.

import asyncio
 
async def main():
    print('Hello')
    await asyncio.sleep(1)
    print('World')
 
asyncio.run(main())

Scatter gather:

import asyncio
 
async def task(name, delay):
    await asyncio.sleep(delay)
    print(f"Task {name} finished")
 
async def main():
    tasks = [
        task("A", 1),
        task("B", 2),
        task("C", 3)
    ]
    await asyncio.gather(*tasks)
 
asyncio.run(main())

Asynchronous IO is perfect for situations where the problem is best solved with monitoring a large number of unix file descriptors (this can be sockets, files, eventfd descriptors etc) for events while sleeping most of the time. You have one thread, but can monitor multiple event sources.

Unit testing

Python provides very powerful unit testing framework that you can use to test your code (have a look at pytest).

import unittest
 
def add(a, b):
    return a + b
 
class TestAddition(unittest.TestCase):
    def test_add(self):
        self.assertEqual(add(1, 2), 3)

Notice that in this case we have tests directly embedded in the file. While you can do this (you can run the above code using pytest file.py), normally you would place your test cases into tests folder and prefix files containing test cases with test_ prefix.

File Handling

Reading and writing files is straightforward in python.

with open('output.txt', 'w') as f:
    f.write("Hello, world!")

with open('input.txt', 'r') as f:
    data = f.read()
    print(data)

Context Managers

This is how you implement objects that can be used in a with block.

class MyContextManager:
    def __enter__(self):
        print("Enter the context!")
        return self
 
    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Exit the context!")
 
with MyContextManager() as manager:
    print("Inside the context!")

Debugging

You can debug python directly using debugging helpers.

import pdb
 
def divide(a, b):
    pdb.set_trace()
    return a / b
 
print(divide(1, 0))

Run using python -m pdb my_script.py. Even if you run your script using python3 my_script.py the script will pause at the line before the return and you will be able to inspect your variables.

Profiling

You can use cProfile to profile your code in python:

import cProfile
import re
 
def regex_operations():
    return re.compile("foo|bar").match("bar")
 
cProfile.run('regex_operations()')

Design patterns: Singleton

class Singleton:
    _instance = None
 
    def __new__(self):
        if self._instance is None:
            self._instance = super(Singleton, self).__new__(self)
        return self._instance
 
singleton1 = Singleton()
singleton2 = Singleton()
print(singleton1 is singleton2)  # True

Packaging

You should always package and version your python code into python packages that can be installed with pip. This ensures orderly releases and easy version management.

When you crate a pyproject.toml file in the root of your project you can then build a pip package using python -m build command.

Useful Libraries

NumPy: numerical analysis and AI programming.

import numpy as np
a = np.array([1, 2, 3])
print(a + 1)

Pandas: useful for data manipulation and analysis.

import pandas as pd
df = pd.DataFrame({'Name': ['John', 'Anna'], 'Age': [28, 24]})
print(df)

Matplotlib: for plotting graphs.

import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [5, 7, 4])
plt.show()

SQLite: for simple database

import sqlite3

connection = sqlite3.connect('example.db')
cursor = connection.cursor()

# Create table
cursor.execute('''CREATE TABLE IF NOT EXISTS stocks
                  (date text, trans text, symbol text, qty real, price real)''')

# Insert a row of data
cursor.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")

# Save (commit) the changes
connection.commit()

# Close the connection when done
connection.close()

Flask: easy creation of web endpoints.

Flask is a very powerful library for creating network endpoints and apis:

from flask import Flask
app = Flask(__name__)
 
@app.route('/')
def hello_world():
    return 'Hello, World!'
 
if __name__ == '__main__':
    app.run()

For outgoing requests you can use requests library:

import requests
 
response = requests.get('https://api.github.com')
data = response.json()
 
print(data)

TKinter: quick graphical interfaces

import tkinter as tk
 
root = tk.Tk()
root.title("Simple GUI")
root.geometry("300x200")
 
label = tk.Label(root, text="Hello, Tkinter!")
label.pack()
 
root.mainloop()

nltk: natural language processing

import nltk
nltk.download('punkt')
from nltk.tokenize import word_tokenize
 
text = "Hello there, how are you?"
tokens = word_tokenize(text)
print(tokens)
# Result:
['Hello', 'there', ',', 'how', 'are', 'you', '?']

Tensorflow: machine learning and model identification

import tensorflow as tf
 
# Define a Sequential model with a single dense layer
model = tf.keras.models.Sequential([
  tf.keras.layers.Dense(units=1, input_shape=[1])
])
 
model.compile(optimizer='sgd', loss='mean_squared_error')
 
# Training data: y = 2x - 1
xs = [-1.0, 0.0, 1.0, 2.0, 3.0, 4.0]
ys = [-3.0, -1.0, 1.0, 3.0, 5.0, 7.0]
 
# Train the model
model.fit(xs, ys, epochs=500, verbose=0)
 
# Predict the value of y for x = 10.0
print(model.predict([10.0]))
# Result:
[[18.98]]

PuLP: solving linear programming problems. This allows you to define constraints and then solve the problem while adhering to all specified constraints.

from pulp import *
 
# Define the problem
prob = LpProblem("Simple_Problem", LpMaximize)
 
# Define the variables
x = LpVariable("x", 0, None)  # x >= 0
y = LpVariable("y", 0, None)  # y >= 0
 
# Objective function
prob += 3*x + 2*y, "Objective"
 
# Constraints
prob += x <= 20, "Constraint_x"
prob += y <= 30, "Constraint_y"
prob += x + y == 30, "Constraint_sum"
 
# Solve the problem
prob.solve()
 
# Print the results
print(f"Status: {LpStatus[prob.status]}")
for v in prob.variables():
    print(f"{v.name} = {v.varValue}")
# Result:
x = 20.0
y = 10.0

Streamlit: a faster way to build web application.

import streamlit as st
 
# Write a title and some text to the app:
st.title('Hello, Streamlit!')
st.write('This is a simple Streamlit application.')

To run do: streamlit run file.py

The possibilities are endless.

SLIP Networking DevOps