In this last episode of this course, I talk to Itamar Turner-Trauring who created the website PythonSpeed and spent a considerable time on finding ways to make Python code faster and more efficient. Python and its ecosystem also have great tools how you can measure performance.

Links:

• https://pythonspeed.com a set of articles and recommendations on how to improve your performance
• https://blog.sentry.io/python-performance-testing-a-comprehensive-guide/ a general blog post on performance testing
• https://uwpce-pythoncert.github.io/SystemDevelopment/profiling.html
  • https://uwpce-pythoncert.github.io/SystemDevelopment/index.html
• https://en.wikipedia.org/wiki/Computer_performance
• https://python-102.readthedocs.io/en/latest/performance.html
• https://docs.python.org/3/tutorial/datastructures.html
• https://www.green-algorithms.org
• https://doi.org/10.1145/356635.356640 Donald Knuth's paper on over optimisation
• https://wiki.python.org/moin/TimeComplexity
• https://blog.jetbrains.com/dataspell/2023/08/polars-vs-pandas-what-s-the-difference/ comparing Polars with Panda

Profiling tools

• https://pyinstrument.readthedocs.io/en/latest/
• https://docs.python.org/3/library/profile.html
• https://docs.python.org/3/library/time.html the time function in Python
• https://docs.python.org/3/library/timeit.html another function to measure time in Python
• https://jiffyclub.github.io/snakeviz/ a graphic profile viewer
• https://bloomberg.github.io/memray/ flexible memory profiler
• https://github.com/benfred/py-spy
  • https://www.wrighters.io/profiling-python-code-with-py-spy/
• https://github.com/P403n1x87/austin-python The Python wrapper for the Austin profiler

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

In this episode I talk to Jeremiah Miller - a software engineer - and Max Albert - a research software engineer in the research software group at the University of Southampton about refactoring and design patterns.

• https://en.wikipedia.org/wiki/Design_Patterns
• https://en.wikipedia.org/wiki/Spaghetti_code
• https://www.distributed-systems.net/index.php/books/ds4/ book by A Tanenbaum
• https://refactoring.com The Refactoring book by Martin Fowler
  • https://martinfowler.com/architecture/
  • https://martinfowler.com/eaaDev/
  • https://martinfowler.com
• https://www.patternlanguage.com the original book by Christopher Alexander on design patterns in architecture - for towns, cities etc. This book inspired software engineers to define a set of design patterns on how to structure code
• https://refactoring.guru/design-patterns Max recommended Sandi and her tips and recommendations on coding
• https://sandimetz.com/99bottles the idea of making things as identical as possible to sniff out design breaks or changes
• https://refactoring.guru/design-patterns another website on design patterns and refactoring

Some books:

• Design Patterns - Elements of Reusable Object-Oriented Software Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides, Addison-Wesley, 1995, ISBN 0-201-63361-2
• Enterprise Integration Patterns Gregor Hohpe, Bobby Woole, Addison-Wesley, 2004, ISBN 0-321-20068-3

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

In this episode I talk to Monika Byrne Svata, Agile Delivery Manager at UCL about project management and in particular Scrum and Kanban. GitHub has features, like Issues and Projects that can help you stay on top of your software projects.

Links

• http://www.linkedin.com/in/monikabyrnesvata Monika's LinkedIn profile
• https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects GitHub projects documentation
• https://www.atlassian.com/software/jira JIRA
• https://trello.com/home Trello
• https://agilemanifesto.org Agile Manifesto
• https://www.scrum.org/resources/what-scrum-module Scrum, a popular agile methodology
  • https://www.agile42.com/en/blog/scrum-history Scrum history
• https://www.atlassian.com/agile/kanban Kanban methodology
  • https://en.wikipedia.org/wiki/Kanban a bit of background to Kanban
• https://beza1e1.tuxen.de/waterfall.html Waterfall methods
• https://digital.ai/resource-center/analyst-reports/state-of-agile-report/ annual report on all things agile

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

Documentating software is part of the life of software engineers. But what kind of documentation do we need? In this episode I take you through three levels of documentation: the basic README and LICENSE files everyone should have, how to be good at writing git commit messages and using tools to turn your source code comments into browsable documentation.

Links

• https://www.sphinx-doc.org/
• https://pdoc.dev/docs/pdoc.html
  • https://docs.python.org/3/library/pydoc.html
• https://wiki.python.org/moin/DocumentationTools
• https://peps.python.org/pep-0287/ reStructuredText Docstring Format
• https://github.com/matiassingers/awesome-readme some README examples
  • https://dev.to/documatic/awesome-readme-examples-for-writing-better-readmes-3eh3
• https://www.gitkraken.com/learn/git/best-practices/git-commit-message GIT commit messaging
• https://www.warp.dev/terminus/git-commit-history Git Commit history

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

Testing your software is part of development. In this episode I talk about different types of testing, automated tests, legacy code etc. My guest is Stef Piatek from UCL who tells us how he approaches testing in his daily work.

Links

• https://docs.python.org/3/library/unittest.html
• https://docs.pytest.org/
  • https://docs.pytest.org/en/7.1.x/how-to/monkeypatch.html PyTest Monkeypatch
• https://agiledata.org/essays/tdd.html test driven development TDD
• https://en.wikipedia.org/wiki/Extreme_programming
• https://joss.readthedocs.io/en/latest/review_criteria.html criteria for open source software reviews, which includes a section on testing
• https://www.freecodecamp.org/news/a-practical-guide-to-start-opensource-contributions/
• https://docs.github.com/en/actions GitHub Actions
• https://martinfowler.com Martin Fowler's great web site. Also look out for his book:
  • Working Effectively With Legacy Code - Michael Feathers, 2004, ISBN: 8601400968741

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

In this episode I'll be discussing data formats such as CSV, JSON and YAML. My guest is Nick Radcliffe from Stochastic Solutions and the Uni. Edinburgh. Nick's expertise is in data science and he has a lot to share about data, data formats and how to use them.

Links

• https://www.linkedin.com/in/njradcliffe/ Nick's LinkedIn profile
• https://en.wikipedia.org/wiki/Comma-separated_values CSV formats
• https://www.json.org/json-en.html JSON
  • https://json-ld.org JSON for linked data
  • https://json-schema.org JSON schema
• https://yaml.org YAML
• https://parquet.apache.org Parquet by Apache
• https://hdfgroup.github.io/hdf5/ HDF5

Libraries

• https://numpy.org
• https://scipy.org
• https://scikit-learn.org/stable/
• http://www.tdda.info test driven data analysis

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

Python is one of the most widely used programming languages in research and science. How did it come to that. And what makes Python special? It's something I discuss with my guest in this episode, Robin Wilson, who also takes us through some of the less favourable or more complex aspects of the language.

Links

• http://www.rtwilson.com Robert Wilson
  • https://blog.rtwilson.com Robert's blog posts
• https://www.python.org
• https://anaconda.org Anaconda Python distribution
• https://jupyter.org all things Jupyter
• https://ipython.org iPython
• https://inference-review.com/article/the-origins-of-python
• https://en.wikipedia.org/wiki/Literate_programming

Python libraries for science

• https://numpy.org
• https://scipy.org
• https://pandas.pydata.org
• https://scikit-learn.org/stable/

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt

In this episode we look into more essential Git commands, such as branching and merging. Branching and merging are key concepts that help you develop code or even text documents in a team. They help you maintain different versions of files and work on them independently.

Another element of collaborative working is provided by GitHub: the pull request. Pull request are a great way to do code reviews, which avoids introducing bugs and also learn from each other.

In my conversations Sam and Eirini talk about these key features and their experiences with it.

Links:

• https://git-scm.com/ Git again
• https://martinfowler.com/articles/branching-patterns.html Martin Fowler's blog post on branching patterns
• https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests explains the mechanics of pull requests in more detail
• https://github-pages.ucl.ac.uk/rsd-engineeringcourse/ Lecture notes

Don't be shy - say Hi

This podcast is brought to you by the Advanced Research Computing Centre of the University College London, UK.
Producer and Host: Peter Schmidt