Python

exeuctable pseudo code

• high-level + multi-purpose
• readability > speed
• object oriented language (OOL)
• very popular
  • many resources exist
  • community developed packages and modules

two-language problem

• prototyping in slow language
• deployment needs translation to fast language

Virtual Environments

every project deserves its own environment

• isolate project from operating system
  • prevents breaks when installing packages somewhere else
• shareable
• can easily be done with venv package

venv

python -m venv <name/your/env>                      #creation
source <path/to/env>/bin/activate                   #activation

python -m pip install <name of package>             #installation
pip install <name of package>                       #installation
pip install -r <path/to/requirements.txt>           #installation
pip list                                            #listing

pip freeze > <path/to/requirements.txt>             #sharing

choose your own package manager (I recommend pip).
Whatever you do, try NOT to mix package manager (i.e., conda + pip)

Action!

in your copy of https://github.com/swincas/cnc-computing-course.git

1. initialize a new virtual environment
2. activate it
3. install all the packages needed for the remainder of the course using the requirements.txt file
4. install the cn5_pkg/ by calling pip3 install . from the project root

Good To Know

• you can install your own packages (if they are configured accordingly)
  • need a setup.py or pyproject.toml
• -e installs the file in editable mode
  • any changes you make to the source will be reflected in the installed version

pip3 install -e <path/to/your/package> 

• keep your venv clean with pipreqs
  • creates requirements.txt that only contains packages that get used in your project
    • ignores depdencies of these packages

    pip3 install pipreqs    #install
    pipreqs . --force       #create (force to overwrite existing file)

Useful Packages

NumPy

• main package for numerical computations
• philosophy: vectorization > parallelisation
• row-indexed
• need for speed
  • compile to C
  • consistent data types
  • views > copies
• enhancements: np.einsum(), Einops
• examples

make use of @np.vecrtorize to increase speed (example)

matplotlib

• Python's plotting library
• highly customizable

render-hieararchy:

1. Canvas
2. Containers: Figure, Axes
3. Artist: lines, markers, text, ...

Design Guidelines

• understandable without extra explanation
  • caption should guide the user, not substitute the plot
• readable
  • font-size
  • clutter
• understandable without colours
• should present key result

most paper readers look at the plots before reading the text.
the plot shall sell your result/idea

Customizing matplotlib

Approach 1: style sheets

• default matplotlibrc file
• use via the following
• examples

plt.style.use("<path/to/yout/file.mplstyle")

Approach 2: Function + plt.rcParams

• see this repo for examples
• see here for usage demos
• examples

Data Frames

pandas

• processing tabular data efficiently
• loads of convenience methods and functions
• examples

Polars

• pandas on steroids
• not as many conveniences, but but efficient (memory, speed)
• examples

astropy

• developed specifically for astronomy
• useful integrations
  • astronomical data-formats
  • units
  • ND data processing
  • coordinates (SkyCoord, World Coordinate System),
  • cosmology (FlatLambdaCDM)
• examples

Other Useful Packages

• Scipy
  • variety of tools useful in scientific computing
  • see stats-course by Chris Blake
• joblib
  • see later session
• mpi4py
  • see later session
• seaborn
  • statistical data visualization
• scikit-learn
  • (shallow) machine learning in Python

Good Practises: logging

• print() can go haywire especially on supercomputer
• logging allows customization of displayed messages
• examples

import logging
logger = logging.getLogger()                    #root logger
local_logger = logging.getLogger(__name__)      #local logger
logging.basicConfig(level=logging.WARNING)      #only show warnings
local_logger.setLevel(logging.DEBUG)            #all logged messages

logger.info("hidden")
logger.warning("shown")
local_logger.info("not hidden")
local_logger.warning("shown")

Good Practises: Unit Tests

• unit tests ensure that your code does not break
• crucial for collaboration

investing 30 min to write a good test prevents hours of headache down the line

• useful module: pytest
• see cn5_test.py for an example

#running a test
pytest <path/to/test/directory>

#testing cn5 package (run from project root)
pytest ./test