This is the first in a series of articles on calculating statistics from datasets using Python. This article’s code will create a simple Python list of random data and use built-in functions and the statistics module.

The Project

This article has a single Python file, pyliststats.py, which you can clone or downloaded from the GitHub repository. Code from future articles will be uploaded to the same repository.

The Code

This is pyliststats.py.

At the top of the file random and statistics are imported. In main we firstly create and populate a list of random numbers.

In lines 17 to 20 four of Python’s built-in functions are called. The function names are self-explanatory and calculate the length, minimum, maximum and sum (total) of the values in the iterable argument. I have omitted exception handling but an empty iterable will raise a ValueError so add exception handling to production code.

The methods in the statistics module are more sophisticated and require some elaboration. These raise a StatisticsError with empty iterables.

Means

The two means used here are the arithmetic means, what most people mean (!) by “average”, ie. add up all the numbers and divide by the count. The fmean method converts all values to floats (although our data is all floats anyway) and is more efficient than mean. There may be minor differences in the return values of the two functions.

Mode

The most frequent value, or the first of these if there is a “draw”.

Standard Deviation

The mean is just a single value which gives no indication of the range of the data; we could calculate the “mean deviation from the mean” or how much, on average, values are above or below the mean but the standard deviation is an ultimately more useful indicator. We know that our data ranges from 1 to 1000 and if you run the code a few times the pstdev is typically in the high 200s which gives a reasonable idea of the spread. Note that I have used pstdev or population standard deviation. This is slightly different from the stdev method which should be used for a sample of data.

Medians

The median is the middle value of the sorted data and is somewhat more meaningful that the arithmetic mean as it is less affected by skewed data, ie. when values are more heavily concentrated at the bottom or top ends. The statistics module spoils us with no less that four median methods to choose from although I have only used three so far.

• median: the middle value of an odd number of values or the mean of the two middle values if the count is even

• median_low: if the count is even the lower value is returned

• median_high: if the count is even the higher value is returned

Quantiles

If you sort the data and divide it into chunks with the same number of values in each then the quantiles are the values at the dividing lines of these chunks. The most common are quartiles which divide the data into four groups and deciles which divide it into ten. These give a reasonable indication of the spread of data and the upper and lower quantiles may be more meaningful that the minimum and maximum values if there are outliers, ie. a few values a long way from the typical range. (For this article I have used random.uniform so the data is far better behaved than you would typically find in the wild so there aren’t any values you could consider to be outliers.)

The quantiles method returns a list of the border values, the default number of quantiles being 4 for which we get three values. I have also called the method with n=10 for deciles in which case we get nine values. In each case the middle value is of course the same as the median. In a future article I will show how quantiles can be represented graphically.

Running the Program

Run the program like this:

python3 pyliststats.py

Here we have the output. Don’t bother reading it: this article is more about the process than the end results.

Leave a comment

Conclusion

The various functions demonstrated in this article’s code might be all you need especially for small datasets, but they are rather rudimentary and, if you are learning statistics, completely opaque. You throw a list at a function and it gives you back a number or another list with no indication of their meaning or how they are calculated.

For the next article in this series I will write a class that calculates the stats we have looked at so far, primarily to raise the veil of mystery behind them but also because it might be a faster way of doing things. Any possible performance increases will be investigated in Part 3.

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

I have already written a couple of introductory articles on the Pillow image library and another on using the library to read EXIF data. These articles are linked below.

An Introduction to the Python Pillow Image Library
The Python Pillow Image Library part 2
Reading EXIF Data with Python and Pillow

The first article includes a demonstration of how to save an image opened with Pillow, either to create a copy or save changes made by Pillow. This is done by passing a filename to the save method but saving any EXIF data which may have existed in the original file needs a bit more work.

The Code

The code for this article consists of a single file called pillowsaveexif.py which lives in the Github repository. This repository also contains the code for my other Pillow articles and photo.jpg, the big orange flower.

This is the entire code in pillowsaveexif.py.

Imports

Here we just import Image from the PIL library. (As I mentioned in an earlier article PIL was inherited from Pillow’s ancestor.)

The main function

The openfilepath variable is a file with EXIF data. This is passed to two functions which will save copies without and with the EXIF - note the filenames.

save_NO_exif and save_WITH_exif

The code in save_NO_exif is essentially the same as in the first article of the series in that any EXIF data is lost.

In save_WITH_exif we first check the image’s info property, a dictionary, to see whether there is a key called “exif”. If so then all we need to do is pass it to save as an optional exif argument. Simple!

If the info dictionary does not contain any EXIF then we just call save with a filename.

The Path to Success is Paved With Failure

As an alternative to the if/else I tried this:

exif = image.info.get(’exif’) or None

And then passed the exif variable to save whatever its value. This of course worked fine for images that have EXIF data but passing None as the exif argument caused an exception and, worse, the file to be corrupted. I thought I would mention this just in case you had the same idea, and also as a reminder that neat, tidy and working code is often the result of much trial and error.

Trying It Out

Run the program like this:

python3 pillowsaveexif.py

There is no terminal output (unless you get an exception for some reason) but you’ll find a couple more image files have been created, photo_no_exif.jpg and photo_with_exif.jpg. You might like to verify their respective absence and presence of EXIF data.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

In Part 1 of this series I wrote a short Python program that printed sample tables with Unicode border characters. In Part 2 I get to the central purpose of this series: writing a function to print a table with actual headings and data.

The files for this series live in their own GitHub repository. In Part 2 we add a couple more files:

• tableizator.py

• tabledemo.py

The Code

The code to generate a table is in the tableizator.py file. The process consists of four main steps, each of which is carried out by a function call in tableize. The steps are:

• Calculate the column widths from the longest item in each heading/column

• Create four types of horizontal border string

• Print the headings

• Print the data

As tableizator.py is fairly long I’ll look at it piece by piece.

The tableize Function

After importing a few members of the typing module four functions are called in tableize, as described above.

The __calc_col_widths Function

Firstly the heading widths are obtained with a list comprehension and then within a nested loop these are overwritten if any data is longer than its heading. This means that for each column we know the widest value. The list is returned as a tuple.

The __create_border_strings Function

This function is a lot simpler than it looks because it has to do everything four times. I’m not entirely satisfied with this and may revisit the code one day and try to streamline it.

For each of the four types of horizontal border we need to carry out four tasks:

• Add the leftmost character

• Iterate the columns and append the appropriate number of characters which go above or below each item

• Append an appropriate intersection character

• Append the appropriate end-of-row character

Having done that the strings are returned as a dictionary.

The __format_heading and __format_data Functions

The __format_heading function is just a single line but I felt it was worth putting in a separate function to allow for more complex enhancements without cluttering up the __print_headings function.

The __format_data function is a bit more involved as the alignment depends on the data type, and again this could expand to something more sophisticated.

The __print_headings and __print_data Functions

The __print_headings function assembles a single string for the headings and their border characters and then prints it sandwiched between heading_top_border and heading_bottom_border, these retrieved from the dictionary created by __create_border_strings

The __print_data function is basically the same but of course has to iterate rows as well as columns. The border string printed in each iteration depends on whether the last row has been reached.

A Demo Program

This is a simple bit of code to try out our Tableizator. It just creates list of headings and data and throws it to tableize to do its stuff.

Run the program with this command:

python3 tabledemo.py

This is the result.

Conclusion

I am aware that this little project is very minimalist and that there is a potentially huge number of enhancements. A few that spring to mind are:

• More sophisticated formatting

• Specifying actual or maximum column widths

• Specifying which columns to include from a larger data set

• Alternative border characters (various options were shown in Part 1)

• A curses version with colours and other formatting

If anyone wishes to fork the project and enhance it then please go ahead. I’d be very interested to see what you come up with.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

NumPy is a large and complex library and most users will only need a small subset of its functionality. It is very well documented here but I often find it useful to keep snippets or bits of sample code, especially for features I need occasionally but not frequently enough to commit to memory. The code for this article consists of a Python file containing a number of such snippets which can be copied, pasted and edited for specific situations. You may or may not find them useful but that is not the point - the purpose of this article is to encourage and inspire you to create and maintain your own cheatsheet.

If you are not already a NumPy user you might like to read my introductory NumPy Reference.

This project consists of a single file called numpycheatsheet.py which you can find in the Github repository. Please feel free to edit and adapt it to your own needs. Each NumPy feature covered by this article has its own function in the file; just uncomment the one you want to run in the main function. I have chosen a selection of common tasks which I feel will cover the majority of what a typical NumPy programmer will need on a day to day basis.

The code snippets all assume NumPy has been aliased as np, the usual convention.

The Code Snippets

Version

You might occasionally need to check the version of NumPy you have installed. To do this simply grab the np.__version__ property.

Creating Arrays

There are several ways to create a NumPy array including from a Python list or with the zeros or ones methods which create an array filled with 0s or 1s respectively.

Statistics

NumPy contains methods to calculate a variety of statistics including arithmetic mean, sum, minimum and maximum etc. These can be called on NumPy (np) like this np.sum(data) or on a NumPy array like this data.sum().

Array Information

There are a few methods to get information about an array including ndim (number of dimensions), size (total number of elements) and dtype (data type).

unique

The unique method returns a sorted array of unique values, ie. it “de-dupes” the values.

csv_files

The savetxt and loadtxt methods respectively save an array to a CSV file and create an array from a CSV file. As with any code that attempts to interact with the outside world, even the local file system, calls to these two methods should be in try/except blocks.

Data Types

A NumPy array has a data type depending on its contents. In general you can let NumPy figure out the type for itself but you can specify the type when creating an array and, as we saw in the info function, you can read the dtype property.

Iterating an Array

It’s sometimes useful to explicitly iterate the values in an array, for example when printing the values in a format other than the default. You can use a normal for/in loop or obtain an iterator with the nditer method. Note that operations on an array’s elements rarely if ever require an explicit loop and can be performed on the array as a single entity.

where

The where method returns an array of indices of the values in the input array satisfying the where clause; this can then be used to retrieve the actual values you want. I have included an alternative syntax with these two operations combined into a single line.

Random Numbers

NumPy’s pseudo-random methods are useful for generating test or demo values, and include a handy Gaussian or normal distribution generator.

Arithmetic

Possibly NumPy’s most useful feature is the ability to carry out operations on all elements of an array in a single line with no explicit looping. The code in the arithmetic function demonstrated adding, subtracting, multiplying and dividing two arrays as well as one array and a scalar. NumPy has methods called add, subtract, multiply and divide but I much prefer the good old fashioned +, -, * and / symbols.

linspace

This is another way of creating an array which I thought deserved its own function. The linspace (linear space) method creates an equally spaced range of values between a minimum and maximum.

reshape

The shape of an array is the number of values in each dimension, and the reshape method rearranges the existing values into the specified number of rows, columns or deeper dimensions.

all and any

These methods return True or False depending on whether all or any of the values in an array satisfy the condition.

Powers and Roots

Numpy provides us with sqrt and pow methods for square roots and powers but you can also use the standard Python ** syntax which I prefer.

The n^th root of a number is the number to the power of 1/n, so for example the square root of a number is the number to the power of 1/2 or 0.5. I have included a demonstration of this in the sample code. NumPy also has a cbrt (cube root) method but if you are unlucky enough to need and other roots you will have to use the power of reciprocal technique.

stacking

The ...stack methods combine two or more arrays vertically (vstack) or horizontally (column_stack).

Appending

The append method creates a new array with the specified iterable or scalar appended. This is not an in-place operation so the return value must be assigned to a variable. In the code I have overwritten the original arrays with the return values of append but if you do not do this the original array is unchanged.

Reversing with flip

The flip method reverses an array but works across all dimensions, so for example a 2-dimensional array is reversed both horizontally and vertically.

Conclusion

I hope you found something useful in the sample code in numpycheatsheet.py and more importantly were inspired to create your own NumPy cheatsheet.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

If you have ever attempted to draw a Bezier curve in a graphics application such as Inkscape you may have found it baffling and frustrating. Bezier curves are defined relative to control points but how these affect the curve isn’t always obvious. In this article I will write Python code to illustrate how the curve is derived from the points and I hope you find the output instructive.

As Confucius didn’t say “a picture is worth a thousand words” so to start I’ll show you a Bezier curve with its secrets exposed. I’m sure you will find it instantly enlightening.

This is an annotated version of the Matplotlib output created by this article’s code. The points P₀, P₁ and P₂ are key to the whole process of creating a Bezier curve. In this article I will look at curves with three control points; these are known as quadratic Bezier curves. These are the salient points to remember:

• P₀ and P₂ are the ends of the curve

• P₁ governs the behaviour of the curve between its end points

• The start and end points of the intermediate lines are equally spaced along the lines between the control points L₁ and L₂

• Given enough intermediate lines the outermost intersection points give an acceptable illusion of a curve

The code to create this illustration contains data structures to represent the points and the inter-point lines which are then used to create a data structure of the intermediate lines. This last data structure is then plotted by Matplotlib.

The Project

This project consists of the following files which you can find in the GitHub repository.

• namedtuples.py

• bccalc.py

• bcplot.py

• bcdemo.py

In Part 2 I will extend the code to calculate and plot just the points along the curve. For Part 3 I will develop an interactive Bezier curve interface using PyGame.

The Code

namedtuples.py

In this file three named tuples are defined to make the code that calculates points, lines and the distances of the intermediate lines clearer.

bccalc.py

This is where the hard work is done. The arguments to the create function are a tuple of points - specifically named tuples from the previous file - and the number of intermediate lines to draw. The points are split out into separate variables purely to shorten the code using them.

The variables l1 and l2 connect the control points, and lines is an empty list to which new lines will be appended as they are created. Next we calculate the horizontal and vertical distances between intermediate lines along both the existing lines connecting the control points. Finally a for loop iterates the specified number of intermediate lines, calculating the coordinates of their endpoints and appending each new line to our list.

bcplot.py

This is a very straightforward usage of the Matplotlib library. If you aren’t familiar with Matplotlib you might like to read my Matplotlib Reference article.

The plotlines function takes as arguments a list of lines created by the code in bccalc.py and a color. It then iterates the lines, calling Matplotlib’s plot method on each. Note that we need to set the aspect to equal as the units of each axis are of the same type; without this the plot would probably be distorted.

bcdemo.py

Now we just need to try the code out.

After importing the three previous files three points are created and passed to bccalc.create, the return value of which is of course a list of lines which are then passed to bcplot.plotlines.

Run the program with this command:

python3 bcdemo.py

This is the end result.

You may want to try out a few other combinations of control points by editing bcdemo.py or uncommenting the three already in the main function.

Conclusion

The code above gives a clear and intuitive demonstration of how a bezier curve is put together and how the three control points do their job. For the next part of this series I’ll isolate the outermost intersection points so that we can plot just the curves without their “scaffolding”.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

Amongst the 160,000 or so Unicode characters are a set of forty which can be used to form borders around data printed in a table format. The ultimate aim of this project is to create a function which will use these characters to generate a table from iterables but in Part 1 I will write code which prints the characters to provide a reference to what is available, and also print a few examples of how they can be used.

The project so far consists of a single file called symbols.py which you can find in the GitHub repository. Further files from this project will be added to the same repository.

The Code

This is the symbols.py file.

the main Function

Here we simply call the other two functions, uncommenting/commenting the calls as required.

print_symbols

The hexadecimal values of the border characters are hard-coded into a tuple. Unfortunately the values aren’t in a single sequence so we can’t iterate a range. After printing a heading the tuple is then used in a for/in loop to print the values in a variety of formats, and finally the bottom table border is printed. When this project is finished we will be able to do all the table border stuff automatically.

The code in this function is pretty straightforward but there are a few points to note:

• Hexadecimal values must be prefixed with 0x

• Even though programmed as hexadecimal the values are of course fundamentally binary so can be printed as decimal if required. This is done here using the :0d formatting.

• To print numbers as hexadecimal use :0x (lower case letters) or :0X (upper case letters). I prefer the latter as the letters are the same size as the decimal numbers.

• The chr function gets the character for the specified numeric value.

sample_tables

This function consists of just a pile of hard-coded print statements to show a variety of ways the table border characters can be used.

Running the Program

Run the program like this:

python3 symbols.py

This is the end result. The first eleven characters, which do not have a consecutive sequence of codes, are single-line. The rest, which do have consecutive numbers, are double-line or a mixture of single and double.

In main comment out print_symbols(), uncomment sample_tables() and run the program again. This shows that the characters form a cohesive set for any possible corner, junction or line.

Conclusion

As I mentioned this is just the first stage in a project to create a table-generating function. Stay tuned.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

I recently bought a new pair of binoculars (not the ones in the photo above!) which got me thinking about the way binocular magnifications and lens sizes are specified. These are the key characteristics of a pair of binoculars and are specified in the format [magnification]✕[objective lens diameter in millimetres], for example 8✕30 or 10✕42.

The diameter and therefore area of the objective lenses (those at the very front) govern the amount of light the binoculars capture and therefore the brightness of the image, but the area does not increase in direct proportion to the diameter. This is illustrated by the diagram below.

The two lenses shown have diameters of 25mm and 56mm, the smallest and largest sizes of widely available general purpose binoculars. Even though the larger lens has a diameter just over twice that of the smaller lens the 56mm has an area and therefore light-gathering power about five times larger.

In this article I will write some Python code to calculate the areas of a range of lens diameters, displaying them as a bar graph and a table.

Some Simple Arithmetic

You probably know that the area of a circle is calculated like this, where π is the constant 3.141592653589793 (15dp) and r is the circle’s radius.

Let's modify the formula slightly to allow for the fact that we are working with diameters so need to divide them by 2 to get the radius.

Now let's use the formula with a couple of actual values, 30mm and 40mm, which seem to be the most popular binocular sizes.

As you can see a relatively modest increase in diameter of one third results in an increase in area approaching double.

This isn’t intended to be a binocular buying guide but basically there is always going to be a compromise between performance on the one hand and price/size/weight on the other. Also note that size and weight are not proportional to lens diameter - as a rule of thumb they are more closely proportional to the magnification.

As a final piece of arithmetic I’ll calculate the relative light gathering capabilities of the 30mm and 40mm lenses used in the examples above.

Here we see that the larger lens has 1.78 times the area of the smaller, and the smaller has 0.56 times the area of the larger. This is basically the same information looked at in two ways.

The Project

This project consists of a single file called binocularlenses.py which you can find in the GitHub repository.

The purpose of the code is to take a selection of common binocular objective lens diameters and calculate the actual areas as well as the areas relative to the smallest.

The code uses some simple functionality from the NumPy and Matplotlib libraries. If you are not familiar with these you might like to read my introductory articles.

A NumPy Reference

A Matplotlib Reference

The Code

This is the entire binocularlenses.py file.

After the imports the main function calls the two functions; you can comment/uncomment these to run the ones you want.

The barchart function creates a NumPy array of values representing a range of lens diameters. (This consists of all the integers in the range, most of which aren’t typically used for binoculars.) This is used to create a second NumPy array of areas; note that we can carry out a calculation on an array in one line of code without any explicit iteration.

These two array are used as arguments to a call to the Matplotlib bar method, a few cosmetic properties are set and the show method is called.

If you run the program with this command...

python3 binocularlenses.py

... you will see the following bar chart.

The important thing to note is that the lens diameter roughly doubles but the area increases about fivefold, exactly as shown in the diagram at the beginning of this article.

The table function creates a tuple of lens diameters actually used in binoculars, and then calculates the area of the first; you’ll see why in a moment.

The tuple is then iterated, calculating the area of each current lens. In the print statement we output the diameter and area, and then the area relative to the smallest lens which forms an index baseline value of 1. This forms a handy at-a-glance impression of the relative areas. (If you want to edit the values in the tuple note that they must be in ascending order for this code to work.)

In main comment out barchart(), uncomment table() and run the code again. This is the end result.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

The word isometric means “with equal measurements” and the best known use of the word is probably as a form of “pseudo-3d” graphics without perspective. This means that a particular distance will appear on screen the same size irrespective of its supposed distance from the viewer. You may already be familiar with isometric projections even if you didn’t know the term as they have been used for decades in video games, and are far simpler to calculate than true 3d coordinates.

In this article I’ll write a very minimalist introduction to the subject which will calculate the two-dimensional coordinates of a three-dimensional cube, and then plot them using Matplotlib. As the title of the article states this is very much a first step and I have a number of ideas for taking isometrics further.

The screenshot below illustrates just what we are going to achieve. The cube on the left is shown with perspective, the red lines being the near face and the green lines being the far face. On the right is the same cube but with the vertices (corners) and edges (lines between corners) calculated to give an isometric projection. The colours remain the same so you can see how the cube has been rotated right and down, although it’s probably better to think of the viewer moving left and up.

The process of calculating the data we need to plot the cube on the right consists of the following steps:

• Create a set of 8 three-dimensional coordinates for the corners or vertices of the cube. These each consist of 3 values: x (horizontal), y (vertical) and z (front to back)

• For each vertex calculate the two-dimensional x and y values. This is the core calculation for the whole field of isometrics.

• From the 2d vertex coordinates create edges consisting of two coordinates each.

• Also using the 2d vertex coordinates, calculate the top, bottom, left and right borders of the three visible faces of the cube.

Having carried out the following steps we can pass the data to Matplotlib so it can do its thing and produce output like that shown above.

The Dreaded Mathematics (Optional)

Calculating two-dimensional x,y coordinates from three-dimensional x,y,z coordinates is an involved process using matrices, some of the matrices used having their values calculated using trigonometric functions. Untangling the trigonometry and matrix arithmetic is way beyond the scope of a programming article although I may write about it on my LaTeX Substack.

I’ll therefore just give a very brief overview of the process to give a flavour of what is going on. Don’t worry if you don’t understand this, and please feel free to skip to the next section if you aren’t interested in the mathematics.

There are three broad stages involved.

Create Constants α and β

These two constants are angles in radians.

Create Three Matrices

These are effectively constant although many of the values are calculated using α, β and trigonometry.

Calculate Two-Dimensional Coordinates

The first matrix, a, contains the three-dimensional coordinates. The second, c, is the product of m1 and m2 from the second stage and a.

Finally, m3 and c are multiplied to create a matrix containing the x and y coordinates. (The others are always 0 and superfluous.)

The Project (Back to Python!)

This project consists of the following files which you can clone or download from the Github repository.

• isometric.py

• isometricplotter.py

• isometricdemo.py

• isometricillustrator.py

The last of these, isometricillustrator.py, creates the side-by-side plots shown above. I won’t show the code here as it’s basically just a load of hardcoded coordinates.

The code uses NumPy and Matplotlib. If you are not familiar with these please read my introductory articles on them.

https://codedrome.substack.com/p/a-numpy-reference

https://codedrome.substack.com/p/a-matplotlib-reference

isometric.py

This is the code for isometric.py which implements the mathematics shown above, and also calculates the vertices, edges and faces of a cuboid.

Constant Values and Matrices

These are the constants which form the first step of the calculations. The symbols are the Greek letters alpha and beta and although I rarely use characters in code that aren’t on my keyboard I decided to do so here to conform to the usual conventions for these calculations.

The matrices m1, m2 and m3 are then created, partly using sine functions and α and β, and partly hard-coded.

xyz_to_xy

This function is at the heart isometric rendering as it converts three-dimensional coordinates to two dimensions, specifically x,y,z to x,y. If you wish to follow the mathematics the code corresponds to the following, repeated from above.

cuboid

This function creates a dictionary of vertices, edges and faces which make up a cuboid with the specified position and size. The separate elements are generated by the various functions described below.

_calculate_cuboid_vertices_3d

This function’s parameters are the ltf (left, top, front) and whd (width, height, depth) arguments from the cuboid function, split out into separate variables to make them easier to use.

Firstly a NumPy array is created to represent a matrix with all values the same, specifically the left top front vertex. The second array contains values specifying how much each point has to be shifted in each axis. By adding these together we end up with a matrix of coordinates of the eight vertices.

_calculate_cuboid_vertices_2d

This function “flattens” the 3D coordinates down to 2D by calling xyz_to_xy on each one and adding the result to a list.

_calculate_cuboid_edges_2d

The twelve edges are created rather tediously by appending the necessary pairs of vertices to a list. I couldn’t discern a pattern to the indices that could be used to streamline the process - if you can then please let me know or, better still, write a shorter and neater version of this function. Thank you.

_calculate_cuboid_faces_2d

Here we create a list of the coordinates of the vertices of the front faces of the cuboid, the ones that can be seen and need filling. The same comment applies as with _calculate_cuboid_edges_2d: if you can make it more concise please do so.

isometricplotter.py

The isometricplotter.py file contains code to plot isometric objects.

The components list is iterated with a for/in loop which in turn iterates and plots the three types of element making up each shape. The vertices are plotted first; note the marker is set to “.” and that the first colour in the list is used. The edges have no markers, a linestyle of “-” and are drawn with the second colour.

Finally the faces are drawn with the fill_between method which, to quote the documentation, “fills the area between two horizontal curves”. Each coordinate argument, x, y1 and y2, consists of two coordinates so together they make up a two-dimensional area to fill. The last of the colours is used, and I have set alpha or opacity to 0.25 so the bits of the cuboid hidden by faces are still visible.

isometricdemo.py

In isometricdemo.py we try out the code listed above.

The isometricplotter’s plot function takes a list of components to plot so first we create an empty list, then create a single cuboid which is appended to the list. Finally this list is passed to plot along with a few other arguments as described above.

The commented line calls a function to create the double plot showing a colour-coded illustration of the rotation process.

Running the Program

Run the program like this:

python3 isometricdemo.py

This is the end result.

Conclusion

The code which calculates two-dimensional coordinates from three-dimensional ones is at the core of isometrics so by coding this process we have laid a firm foundation, as well as put it to good use by drawing a “box”. However, this is only the first step and in future articles I will use the code in PyGame and add further shapes.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

The International System of Units, commonly known as the SI System, consists of seven base units for measuring quantities such as mass, time and electric current. These base units can be combined into derived units to measure a wide range of other quantities.

In this article I will develop a Python class to represent base unit values and associated methods. In a future article I will extend the project to handle derived units.

SI Units

As I mentioned above there are seven base units, as listed below.

There is also a wide range of prefixes representing both positive and negative powers. These are used to create more conventiently sized units although many unit/prefix combinations are not widely used. (When did you last hear anybody say “yottamole”?!)

I won’t list the prefixes here because that would just be boring, but later we’ll see a full list hard-coded in Python.

The Project

This project consists of the following files which you can find in the GitHub repository.

• sibaseunit.py

• siunits.py

The purpose of this project is to create a class that will represent an amount of the required quantity along with any specified prefix. The class will also provide a number of arithmetic and comparison methods which can be safely used with units of the same quantity but different prefixes, for example adding metres and centimetres.

This class will also form the basis of a further project on derived units.

The Code

Let’s get stuck into the code, starting with the SIBaseUnit class in sibaseunit.py.

This isn’t a hugely long class but there’s a lot to discuss so I’ll work through each point one at a time.

quantities

This is a dictionary (wrapped in an immutable MappingProxyType) of the seven base units, each containing a “sub-dictionary” of details.

prefixes

Another MappingProxyType wrapped dictionary, this time for the prefixes including “none”.

__init__

The __init__ method takes a value from the quantities and prefixes dictionaries, and also a value representing the amount of the quantity. This value is stored internally in the base unit, irrespective of any prefix. The conversion is done using the prefixed_value_to_base method which we’ll see later.

As I mentioned in the table above the SI unit of mass is the kilogram, not gram. For the purposes of this project I have assumed common sense prevailed so used the gram for base values of mass. No doubt the SI Police will be after me soon.

Properties

There are two properties, value and prefix. value is settable and converted to the base unit. The prefix property is gettable only.

__repr__ and __str__

The __repr__ method combines the properties into a string representation of the object, and __str__ simply calls string_short which we’ll see next.

string_long and string_short

The string_long method returns the value and the long version of the unit name, for example kilogram or millimetre. The string_short method is similar but uses the abbreviation such as kg or mm.

Arithmetic and Comparison Operators

If you are writing a class that represents objects which can have arithmetic or comparison operators carried out on them then you can easily implement these operations with dunder methods, ie methods with names that start and end with double underscores.

The methods have meaningful names like __add__ for addition and __lt__ for less than. Within these methods you can do whatever you need to achieve the correct result, and here it is necessary in most cases to check that the two SIBaseUnit objects represent the same quantity so we don’t, for example, try to add a mass and a length. This is done by a separate function, __check_quantities. However, the prefixes can be different so values need to be converted to base values before operations are carried out, and converted back to prefixed values before being returned.

Once these dunder methods are implemented we can use symbols such as + and < with instances of the class.

There are two dunder methods for multiplication: __mul__ and __rmul__. The first is for when the object is on the left of the expression and multiplier on the right, and the second is for when they are reversed. This is an example snippet where length is an instance of SIBaseUnit.

length * 4 # __mul__ is called
6 * length # __rmul__ is called

Another slightly mysteriously named dunder method is __truediv__ which is normal rather than floor division. It is used when the / operator is used rather than the // operator. (If you need floor division implement __floordiv__. There was also a __div__ in Python 2.x which is now obsolete.)

__check_quantities

As I mentioned most of the dunder methods work on two instances of SIBaseUnit so need to check the quantities are the same. To avoid repetition this check is split off into a separate function which raises a TypeError if they are different.

output

This method prints out all the information there is on the current instance. You probably wouldn’t use it in production code but it’s useful for demoing and testing.

Converting to and from Prefixed Values

Finally we have a couple of static methods for converting between base values and prefixed values.

Let’s Try it Out

That’s the SIBaseUnit class finished so let’s write a bit of code to show it in action. This is siunits.py.

There are five functions here which are called in main. We can uncomment each one in turn and run it.

quantities

The first is quantities which creates 7 instances of SIBaseUnit, one for each quantity, before calling their string_long, string_short and output methods.

Now run the program:

python3 siunits.py

This is the output, showing only the first object, time.

add_and_subtract

The next function creates two SIBaseUnit objects representing lengths, but with different prefixes: the first is centimetres and the second in millimetres. However, we can still add and subtract them and get the correct results which will have the suffix of the first instance, in this case centimetres.

Uncomment add_and_subtract and run the program again.

add_error

Just to demonstrate what happens if you try to carry out operations on units with different quantities I have attempted to add a length and a mass, within a try/except block which prints any error. As you can see if you run the function the error message is shown.

multiply_and_divide

These two operations work on an instance of SIBaseUnit and a numeric value, rather than two SIBaseUnit objects. This function shows the operations in action including the alternative “flavour” of multiplication where the multiplier comes first.

comparison_operators

Finally let’s look at the six comparison operators. As with addition and subtraction they can safely be used when units have different prefixes.

So now we have a class which can handle the SI base units together with a wide selection of prefixes and operations. Next I’ll combine them into derived units such as metre per second.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

Pretty much any smartphone, tablet or digital camera embeds EXIF data into each photograph it takes, with details of the device itself as well as the individual photo. And just about every piece of software which can display or edit digital images can read and display that data. Or at least attempt to...!

Unfortunately EXIF is not easy to deal with. It would be nice if the data consisted of a string of XML or JSON so that it could be handled using ubiquitous techniques and libraries, but the format itself is rather quirky, made worse by inconsistent implementations by different manufacturers and sometimes different devices from the same manufacturer.

You may have noticed that sometimes even popular mainstream software fails to show sensible interpretations of some EXIF data. Despite that I will have a bash at extracting the EXIF data from images using Python and the Pillow library.

The Project

The Image module of the Pillow library for Python includes a method called _getexif. This returns a dictionary of EXIF data with the keys being numeric codes and the values being in various formats which are not always human-readable.

To obtain something suitable for display to users we therefore need to do a bit of work on this data:

• Obtain the names corresponding to field codes. Pillow provides most of these in the dictionary PIL.ExifTags.TAGS

• Process the data items which are not human readable in some way to make them so

In this project I will write a module with a method called generate_exif_dict. This takes a filename argument and returns a dictionary with textual fieldnames as keys and values which have been processed into some format you might want to let users of your software see.

The solution I have come up with is far from perfect; these are the main imperfections I am aware of:

• Not all tags in PIL.ExifTags.TAGS may be present in an image’s EXIF data (PIL.ExifTags.TAGS has 268 items but my image only has 50 of these)

• Not all tags in an image’s EXIF data may be in PIL.ExifTags.TAGS

• Some data may be in proprietary formats not published by manufacturers and therefore of no use to anyone else

• I have only attempted processing of the more common and useful tags - the more obscure ones have been left as they are

• The module was developed around photos from cameras I have personally used: Nikon, Olympus, Lumix and Fujifilm. Photos taken on cameras or devices from other manufacturers might give inconsistent results

Starting to Code

This project consists of the following Python files which you can clone/download from the Github repository. You will also need an image that has EXIF data in it and the repository includes one of mine, DSC_0502.jpg, but by changing the filename in the code you can use one of your own photos.

• exif2.py

• exifdemo.py

I wrote an earlier version of this code back in 2020 and the then-current version of Pillow returned some numeric values as pairs of integers representing the numerator and denominator of a fraction. For example an aperture of f8.0 was returned as (80,10) requiring a bit of arithmetic to convert it to something more familiar. The latest version of Pillow does that for us so the code in this article is significantly shorter and simpler but the original module, exif.py, is still in the Github repository.

This is exif2.py.

generate_exif_dict

After opening the image with Pillow and grabbing its EXIF data with _getexif() we create an empty dictionary. We then iterate PIL.ExifTags.TAGS and if the key also exists in the image data we store it in the value variable, otherwise value is set to None.

Some EXIF data can be very long, for example the MakerNote, so any data longer than 64 characters - an arbitrary amount - is truncated and an ellipsis added.

An item is then added to the exif_data dictionary, the key being the value from PIL.ExifTags.TAGS (ie the name such as Make or Model) and the value being a dictionary containg the tag number, the raw value and the processed value. The processed value is set to the same as the raw value but some will be changed later.

Finally we pass the completed dictionary to _process_exif_dict to overwrite some of the processed values to a more human-friendly format.

_create_lookups

Some values are encoded as an index, for example MeteringMode can take a value between 0 and 6. The textual equivalents are added to a dictionary as tuples so that we can retrieve them using the encoded values as indexes.

The first two start at 0. Fine. The second two start at 1. Hmm . . . ! I have padded these with empty strings.

_process_exif_dict

After creating a date formatting string and a lookups dictionary the following processing is carried out:

• The 3 dates are strings so are converted to actual dates, although they look identical when printed

• FNumber is prefixed with the letter “f”

• MaxApertureValue is processed in a similar way. Additionally zoom lenses with variable apertures may give us many decimal places so the processed value is formatted to just 1dp

• FocalLength is suffixed with “mm”

• FocalLengthIn35mmFilm is also suffixed with “mm”

• Orientation, ResolutionUnit, ExposureProgram and MeteringMode are all set from the lookup dictionary as described above.

• ExposureTime (ie. shutter speed) converted to a Fraction object so is displayed as “[numerator]/[denominator]”, eg. 1/250.

• ExposureBiasValue (better known as exposure compensation) is suffixed with “EV”, or what I would refer to as stops.

That’s all the processing for now. At a later date I’ll revisit the code and add processing for a few more fields. Please feel free to fork the repository and apply your own processing, especially if you are not happy with the results from images taken on your own camera or phone.

Now let’s write a short program to use the above module and print the results.

This is all very straightforward - we just call exif2.generate_exif_dict and pass the result at the print_exif_dict which iterates the dictionary, printing each name as a heading and then the tag number, raw value and processed value.

Running the Program

Now we can run the program with the command below. I am using the full sized version photo at the top of the page but just change the filename if you want to use your own.

python3 exifdemo.py

This will give us the following output.

This is a truncated version of the output. My photo has 50 pieces of EXIF data but the output shown here is only the top handful. If you use a photograph from another camera manufacturer you will probably see some differences in the fields displayed.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

In the first article of this series I showed how to load an image using the Pillow library and copy the pixel data to a NumPy array. The data can then be used for any purpose you can think of including editing, in which case it can be copied back to a Pillow image and saved.

For this second article I won’t be doing any editing but will use the data to create a set of three histograms showing the distributions of red, green and blue values in an image. If you use any sort of photo editing software you may be familiar with such histograms which are useful for deciding how to adjust colour values. For the moment the histograms I’m creating here are stand-alone and just pop up in a Matplotlib window but in the long term I’m planning on including them in a photo editing application which is in the very early stages of development. (Don’t hold your breath, but I’ll get there one day!)

Image Histograms

If you’re not sure what image histograms represent then this is a brief overview, but if you already understand them just skip to the next section.

A histogram is a very widely used graphical representation of frequency data, using vertical bars to show the frequency of each value or range of values on the horizontal axis. Histograms are used for a wide range of data from people’s heights, exam scores and many others. Another common use is for the red, green and blue values of an image. Colour digital image pixels consists of three values specifying the amount of red, green and blue, each somewhere between 0 (none of that colour) and 255 (the maximum amount of that colour). Histograms showing the frequencies of each colour value look something like this.

Above are the actual Matplotlib histograms produced by the code below for the sample image. The horizontal or x-axis covers a range of 0 to 255, the vertical bars showing the relative numbers of pixels in the image for each value. Note that the frequencies have been normalised, ie reduced to a value between 0 and 1, rather than showing the actual frequencies.

These three histograms are fairly typical in that they tend to be highest across the middle areas while training off towards the ends. Also, the three are of comparable shape which again is typical.

If you have an image which is too light or too dark, or appears to have colour balance issues, then these histograms can help to diagnose the problem and indicate which of the channels (red, green and blue) need to be adjusted and by how much.

The Project

This project consists of two Python files:

• imagehistograms.py

• imagehistogramsdemo.py

All files for this series of articles are in the same Github repository.

For demonstration I have used a photo with the prosaic name of 001.jpg which is included in the Github repository, but if you are going to try out code you might like to use one of your own photos.

This is the 001.jpg photo which I am using for histograms.

This article isn’t intended to be a tutorial on Pillow, NumPy or Matplotlib but the code using each of these is very straightforward. If you don’t have them installed they are all on PyPI and are a quick pip away; these are the links.

https://pypi.org/project/pillow/

https://pypi.org/project/numpy/

https://pypi.org/project/matplotlib/

The Code

This is the imagehistograms.py file.

Let’s look at the various bits of the file.

Imports

This project uses the Pillow library (known as PIL for historical reasons), NumPy and Matplotlib which are imported here.

create_image_histograms

This is the core “pseudo-public” function which calls “private” functions to create and show histograms for the specified image file. Any exceptions are raised to the calling code.

_as_2d

To calculate frequencies it’s far more convenient to have three separate 1-dimensional arrays, one for each of the colour channels. This function firstly splits the array into three separate ones, then transforms them from 2D to 1D. We now have three long arrays containing the colour values for the whole image, which are then returned in a tuple.

_get_rgb_frequencies

All good things come in threes, and most of what this function does is in triplicate. Firstly the frequencies for each colour channel are calculated using the NumPy bincount method. Next we get the highest value for each channel and then the highest overall frequency across all channels.

As I mentioned earlier the histograms show normalised frequencies between 0 and 1. These are calculated for each channel by dividing each set of frequencies by the overall highest. Note here that NumPy arrays can be treated as single entities, so doing something like this . . .

 red_counts = red_counts / max_count 

. . . divides each value in the red_counts array by max_count, returning a new NumPy array with the calculated values. This abstracts away any iteration into single lines of code, while also being many times faster than a loop written in Python would be. Such is the power of NumPy.

_histograms

This function creates Matplotlib histograms from the data created by _get_rgb_frequencies. The second argument is the filename which is just used as a title.

The first couple of lines just create limit values used later to ensure all three histograms cover the same x and y ranges irrespective of the actual data. The possible_values is also used later to ensure the histogram includes all values between 0 and 255 even if some don’t appear in the data.

Next we create a set of 3 plots arranged vertically; note the 3 rows and 1 column. The title is set to the filename, and I have set tight_layout which I think looks better although of course this is only a cosmetic tweak.

The next 6 lines set the x and y limits of each of the three plots. After that the actual histograms are created using the hist method. The data in the counts arrays is used for the weights arguments. Finally we call plt.show() to display the histograms to the outside world.

The Demo Program

Trying out the above only requires a few lines of code. You’ll probably want to edit the filename to use one of your own photos.

Running the Program

Now let’s run the program:

Running the Program

python3 imagehistogramdemo.py

This is the screenshot of the Matplot window again. If you’re not familiar with Matplotlib you might like to play with the buttons and resize the window in the usual way by dragging the edges with your mouse.

I have a number of other articles planned on using image data in NumPy so stay tuned. If you have any suggestions for future articles or have used NumPy with image data yourself please let me know.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

In this project I will write a Python class to represent angles in various units, and provide a selection of arithmetic and comparison operators.

Overview

There is a surprising number of different units used to measure angles. Dealing with them in code can be fiddly for a number of reasons so I have written a class to make the process easier.

The class records not only the angle itself but also the unit you wish to use, for which there is a choice of no less than seven. Whichever unit you choose, the angle value itself is stored in seconds. A second (strictly second of arc or arcsecond) is one sixtieth of a minute (minute of arc or arcminute), which is one sixtieth of a degree. Yes, degrees are sliced up in the same way as hours. The reason for the value being stored in arcseconds is to simplify calculations and comparisons using different instances of the Angle class which may use different units.

In addition to a value and unit the class also provides a selection of arithmetic and comparison operators to simplify using instances in expressions. Due to roundings and slight inaccuracies caused by using pi to calculate radian values, different angles which are intended to be equal might be slightly different, messing up our comparison operators. I have therefore provided a function to test for approximate equality which uses the rather obscure isclose method provided by Python’s math module.

Finally, the __str__ dunder method is implemented to provide a standard and human-friendly string representation of the value and unit.

Now let’s take a brief look at the seven units of angle provide by the Angle class.

• Degrees, minutes and seconds - 360 in a circle, each degree is divided into 60 arcminutes, which are divided into 60 arcseconds

• Radians - see below

• Gradians - 400 in a circle, therefore 100 in a right angle

• Turn - 1 in a circle

• Hour Angle - 24 in a circle

• Point - one unit of the 32 points of a compass

• Quadrant - 4 in a circle

Radians

This article includes the use of radians to measure angles. If you are not familiar with radians you might like to read my article on the topic.

The Project

This project consists of the following files which you can find in the GitHub repository.

• angle.py

• angledemo.py

The angle.py file contains the Angle class, and angledemo.py contains various functions to show the class in action.

The Code

This is the full code listing for the Angle class.

units

This consists of a MappingProxyType (basically an immutable dictionary) with the unit names as keys and inner dictionaries as values. As well as a name the inner dictionaries contain three functions to convert the value to and from seconds, and provide a human readable representation.

The logic associated with each unit is therefore kept together in one neat package, and the whole is easily extensible for other units. You might consider this “design pattern” to be ingenious, or on the other hand you might find it freakish and bizarre!

__init__

This is pretty straightforward, just setting the variables for the value and unit. Note though that the value is converted to seconds using the method associated with the relevant unit.

value Property

As values are stored as seconds the value getter converts seconds to the actual unit before returning it. Similarly, the setter converts units to seconds.

__str__

Here we convert the value from seconds, and then call the unit’s tostring method.

approx_equal

As I mentioned above small inaccuracies can creep into the seconds value used to store all values. This method allows the values of two instances of the Angle class to be compared for equality even if they are slightly different. It uses the math.isclose function provided for us by Python.

The isclose function can take optional rel_tol (relative tolerance) or abs_tol (absolute tolerance) arguments although here I use the default, rel_tol=1e-09, which the Python documentation states “assures that the two values are the same within about 9 decimal digits”.

It’s worth taking a minute to read the math.isclose function’s documentation.

https://docs.python.org/3/library/math.html#math.isclose

Arithmetic Operators

These implement the various arithmetic dunder methods. The actual arithmetic is carried out on the seconds values so that we can carry out calculations with angles of different units. Then the value in the unit of the first angle is calculated, and a new Angle instance returned, again with the unit of the first angle.

Comparison Operators

These are all straightforward one-line implementations, again using seconds to enable angles of different units to be compared.

Let’s Try It Out . . .

The Angle class is now complete so we need some bits of code to try it out.

The main Function

This just has six calls to functions demoing the various aspects of the Angle class. We can comment/uncomment them and run one at a time.

The create_and_output Function

For each of the seven units an Angle instance is created, and the unit name, seconds value, unit value and string value are printed.

For the degreeminutesecond unit a tuple is passed to __init__, the other units just needing a single value.

Run the code like this:

python3 angledemo.py

This is the output of the create_and_output function.

The set_value Function

This is to demonstrate that value can be set, but the new value needs to be in the Angle instance’s unit, in this case a tuple of degrees, minutes and seconds.

Uncomment set_value() in main and run the program. This is the output.

The arithmetic Function

Firstly we create two Angle objects, one with a unit of degreeminutesecond and the other with quadrant. This is to show that it is possible to mix up units.

Next five new Angle objects are created, each the result of a calculation using the first two angles. There are two multiplications, dms * 2 which uses __mul__, and 3 * dms which uses __rmul__.

We then print the original two angles followed by the various results. Uncomment arithmetic() in main and run the program again.

The comparisons Function

Three Angle objects are created and then used with the six comparison operators within print functions. As with the arithmetic function the angles use different units just to show that we can.

This is the output from the comparisons Function.

The approx_equal Function

Firstly I created two Angle objects, the first for 360 degrees and the second for 2π radians, the two being equivalent. If you sneak at the output below you’ll see that the equality operator returns true as both values resolve to the same value in seconds.

However, dividing each by 3 introduces a tiny difference between the seconds values so the == operator returns false, but approx_equal returns true as the two values are within the tolerances.

The convert_units Function

Changing the unit of an Angle object is easy - just do it. As the value is stored as seconds changing the unit has no effect on it, but in future output will of course be shown in the new unit. In the convert_units function we start of with a degreeminutesecond angle, then convert it to quadrant, and finally to gradian. As you can see from the output below the value is retained and output correctly.

I hope you found this article interesting and useful. Even if you have no need for the Angle class a few of the language features such as arithmetic and comparison operators, and the math.isclose function, might be usful for other projects.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

In a previous post I implemented a very simple and very insecure substitution cypher. It is insecure because each letter in the original text is always encrypted the same way, for example the most common letter “e” might always be encrypted as “h”, so if we find that “h” is the most common letter in the encrypted text then we can assume it represents “e”. This can be carried out for all letters, a process called frequency analysis which is the subject of this post.

The Caesar Shift Cypher implemented in the previous post uses a one-to-one mapping between plaintext characters and encrypted characters by shifting along the alphabet by a fixed amount (rolling round to “a” if you get to “z”). If you know or believe that such a cypher has been used you only need to decrypt one letter and you can work out the shift and therefore crack the entire code. Frequency analysis works with a shift cypher but also any substitution cypher where each letter always encrypts to just one other letter, so can be used on more random cyphers.

The frequency analysis process consists of the following steps:

• Create a list of plaintext frequencies, sorted by frequency

• Create a list of encrypted text frequencies, sorted by frequency

• Merge these lists to create an estimated mapping between encrypted letters and plaintext letters

• Use this mapping to create an approximate decryption of the encrypted text

I use the words “estimated” and “approximate” because there is no guarantee the relative letter frequencies in the plaintext original of the encrypted text will match those of the sample text used to create a decryption mapping. However, given texts of reasonable length it is probable that the majority of frequencies will be in the same order, and those that are not will often be just one or two places out enabling us to manually edit our mappings to get a better result.

The Code

This project consists of two Python files, and also a couple of text files which live in this Github repository.

• frequencyanalysis.py

• fademo.py

• the_sign_of_four_plaintext.txt

• encrypted.txt

The first text file is the whole of the Sherlock Holmes novel The Sign of Four and will be used to create a list of letter frequencies which will then be used to attempt a decryption of encrypted.txt.

This is the source code of frequencyanalysis.py.

create_decryption_dictionary

Firstly we read the contents of the two files using the _readfile function, and then call _count_letter_frequencies with each in turn to get dictionaries of the letter frequencies. The dictionaries come to us ready-sorted by frequency which is crucial.

We then create a new dictionary and, within a loop, add key/value pairs with the encrypted letters as keys and plaintext letters as values. We don’t add the actual frequencies as they are irrelevant; we are just interested in the frequency orders.

Finally the mappings dictionary is saved to the specified JSON file.

decrypt_file

This function uses the mappings dictionary in the specified dictionary file to decrypt the encrypted file and save it to the specified path.

Firstly we read the encrypted text into memory, and then the mapping dictionary. Then we create an empty list to hold the individual letters of the decrypted text.

Next we iterate the letters of the encrypted text, getting the ASCII code of the uppercase version of each letter. If it is an actual letter we use it as a dictionary key to get the corresponding decrypted letter which is added to decrypted_list.

Finally we use join to stick all the letters in decrypted_list together into a single string. This is far more efficient than appending letters one by one to a string. Finally we just need to save the decrypted text to a file.

_count_letter_frequencies

This function takes a string and creates a dictionary using the 26 letters of the alphabet as keys and their frequencies as values, sorted by frequency.

After creating an empty dictionary we add the keys to it with frequencies of 0. Next we iterate the text, incrementing the frequencies by 1 for each letter. Finally the dictionary is sorted in descending order and returned.

_readfile

Text files are read three times in the code above so I included a short utility function to simplify things very slightly - it just opens, reads and closes the file before returning the contents. There is no exception handling in any of the above code as this is done by the calling code.

That’s the module finished so we just need a few lines of code to test it. This is fademo.py.

Ignoring the heading and exception handling there are only two lines of code in main, calling the two “public” functions in frequencyanalysis.py. Firstly we call create_decryption_dictionary to create the dictionary and save it as JSON. Then we call decrypt_file to generate decrypted.txt.

The code is finished so run it with this command:

python3 fademo.py

If you open the folder where your files are you’ll find a new file called decrypted.txt which starts like this.

It doesn’t look very decrypted but if we think the encrypted text is a book then it is probable that it starts with the word “chapter”. It actually starts with “msapter” which is promising as only the first two letters are different.

Open decryption_dict.json and swap the following values. (Make sure you swap the lower case values, not the upper case keys.)

• swap m and c

• swap s and h

Comment out the call to frequencyanalysis.create_decryption_dictionary so we don’t overwrite the edited dictionary but use the one we just edited, and then run the program again. This is the new decrypted.txt file.

The text now starts with the word “chapter”, followed by “ursherdockhoduesursherdockhodues”. (The character after the word “chapter” would usually be a number and therefore not encrypted by our simple cypher unless spelled out in words.) If we think the text might be about a character called “Mr Sherlock Holmes” we can easily identify which letters need to be swapped in the decryption dictionary:

• u and m

• d and l

Make these changes in the JSON file and run the program. This is the third version of the decrypted text.

We now have what can be interpreted as a chapter number, a chapter title and the first three words of the chapter itself. After that comes “yhoyaswswalluverulate” which with a bit of thought could be “whowasusuallyverylate” so make the following swap in the JSON file and run the program.

• y and w

This is the next version of decrypted.txt.

We nearly have “whowasusuallyverylate” so make this final edit to the dictionary...

• y and u

...and run the program again. This is the fifth version of decrypted.txt.

It’s not quite perfect - you might want to identify and make a couple more swaps - but it is entirely readable and recognisable as the first chapter of The Hound of the Baskervilles.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

The Euclidean Algorithm is a simple method for finding the highest common factor or HCF (also known as greatest common divisor or GCD) of two positive integers. This is an implementation of the algorithm in Python.

Wikipedia has a comprehensive article on the topic here so let’s get straight to the code. This article’s Python consists of a single file, euclideanalgorithm.py, which you can find in the Github repository.

The main function creates two lists containing pairs of numbers to apply the algorithm to. It then iterates the lists, calling the EuclideanHCF function and printing the result. It also prints each number divided by the HCF as a demonstration or proof.

The EuclideanHCF function replaces b with the modulo (remainder) of a and b, and replaces a with the previous value of b. This is repeated until b reaches 0, leaving a as the required HCF. This is an implementation of the Euclidean division variant of the algorithm which is more efficient than the subtraction method.

Run the program with this command in the terminal:

python3 euclideanalgorithm.py

The program output is:

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

This is a simple Python project implementing a function to calculate the factors of a given integer, ie. all of the numbers which divide into that number. The core function returns a list of the factors of the number argument but as a bonus we can use that to write a generator function consisting of just one line of code!

To calculate the factors of a number n I will use a naïve brute-force method called trial division which attempts to divide n by all numbers up to and including n - if there is no remainder the number is a factor. No number more than n/2 except n itself can be a factor of n so as an optimization I will only run the trial division on numbers up to n/2, and then add n to the factor list. Additionaly, 1 is a factor of all integers so it will be added to the list at the start and we will commence trial division at 2.

This article’s code consists of the following files which you can find in the Github repository.

• factorization.py

• facdemo.py

Let’s first look at factorization.py.

The get_factor_list function creates a list with a single value, 1, which is a factor of any integer so not worth checking. We then enter a loop to perform modular division of n by all numbers up to n/2 - if the result is 0 then the current number is a factor and can be added to the list. Finally we add n to the list and return it.

The function factors is a generator which is a special kind of function returning a type of iterator called a generator object. This function can then be called using for/in syntax to retrieve all values. In this particular case we already have the function get_factor_list to actually calculate factors so within the generator function we can call get_factor_list and pass the list it returns to the iter function. This creates an iterator which we can loop using for/in. All that with just a single line of code.

Now we just need a bit of code to test these functions. This is facdemo.py.

At the top we import factorization and then within main create a list of numbers to factorize. We then iterate this list for the first time, calling get_factor_list on each item and printing the result.

We then iterate the numbers again, this time calling the generator using for/in. This time of course we get the factors one at a time instead of all together in a list.

That’s the code finished so run it with this command:

python3 facdemo.py

The output looks like this:

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

A while ago I wrote a post on estimating Pi using a variety of methods. As a sort of follow-up I will now write a post on estimating e, or Euler’s Number, in Python. I won’t try to summarize what e is or what it is used for as Wikipedia has a comprehensive article on the subject.

The Code

Let’s dive straight into the code which lives in a single file called e.py. (At least you can’t accuse me of using overly long filenames). You can clone/download the source code from the GitHub repository.

This is the first part of the source code.

Near the top of the file I have defined a constant called E15DP which is a string containing e to 15 decimal places. This will be used as a definitive value to compare with values we calculate. The name is all upper case to denote that it should be considered as a constant.

Next we have a few colour constants to output text to the console in various colours.

The main function consists of four function calls to estimate e using various methods. All but the first are commented out for now as we’ll run them one at a time.

The print_as_text function takes a value for e and converts it to a string with 15 decimal places. It then prints it after the definitive value, with correct digits in green and incorrect ones in red.

First Expansion

The first function to calculate e is expansion_1 which implements this formula...

...with a value of n = 100000.

We now have enough to run with this command.

Running the program

python3 e.py

The output is:

As you can see this gives us 4dp of accuracy; you might like to experiment with different values of n.

Second Expansion

Now let’s try another expansion to implement this formula with n = 0.000000001.

Comment out expansion_1() in main, uncomment expansion_2() and run the program again. This is the output.

This gives us 5dp of accuracy but again you can try different values of n.

Newton’s Series

Now let’s move on to the third method, Newton’s Series, for which the formula is:

This means we start off with n = 0, set e to 1/n!, increment n and then keep adding 1/n! to e for as long as we want. (n! means the product of all integers up to and including n, for example 4! = 4 x 3 x 2 x 1 = 24. Incidentally 0! = 1.) This is the function.

Uncomment newtons_series() in main and run the program.

Sir Isaac gives us 9dp of accurancy with a maximum value of n = 12.

Brothers Formula

As with π people are still working on ways to calculate e and I’ll finish off with this impressive recently discovered method, Brothers Formula, devised as recently as 2004.

As with Newton’s Series this is the sum of an infinite series although rather more complex.

The extra complexity pays off though: we only need to count to 8 to get 15dp (possibly more) of accuracy.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

I often use mathematical symbols in Python code which results in me scrabbling around to find the ones I need so I decided to write a simple module to simplify the process, which I will describe in this article.

Overview

The Unicode character encoding system currently (version 17.0, released September 2025) consists of 159,801 characters across 30 sub-categories. The characters likely to be used for mathematical notation are scattered across several categories so for my mathsymbols module I gathered them together into a dictionary, and also provided functions for searching the dictionary and printing out details of each symbol.

With nearly 160,000 Unicode characters it is almost inevitable that I have missed some which could be considered mathematical so if you know of any that you think should be included please let me know.

Once you import the mathsymbols module there are several ways to use it in code which we’ll see later.

The Project

This project consists of the following files which you can clone/download from the GitHub repository.

• mathsymbols.py

• msdemo.py

The Code

This is the mathsymbols.py file.

The symbols Dictionary

This dictionary, which is wrapped in a MappingProxyType to make it immutable, uses brief descriptions as its keys and the characters themselves as values. Note that the descriptions are not the same as those used by Unicode but chosen to be more consistent and easily searchable. This means there is a certain amount of duplication in some keys, for example fractions contain the numbers in both words and digits, eg. fraction_seven_eighths_7_8.

The search Function

This is a simple list comprehension wrapped as a dictionary, returning any key/values with a key containing namelike. I didn’t consider it necessary to provide any wildcard options due to the small number of items in the symbols dictionary.

The print_symbols Function

This prints the supplied dictionary, symbols_to_print, or the entire symbols dictionary if this is not supplied. We can use this function to print a dictionary obtained from the search function.

Trying it Out

Now let’s try it out with the msdemo.py program.

This short program demonstrates printing all symbols, searching and printing the result, and the four ways of using symbols in code.

The line currently uncommented calls print_symbols() without an argument so prints all of the symbols dictionary. Note that I aliased mathsymbols as ms in the import. Run the code thus:

python3 msdemo.py

This will give you the following output. This is truncated but you might like to print all 106 symbols and browse through them to see the sort of stuff available. As you can see we also get the Unicode values in decimal and hexadecimal.

Uncomment the next two lines which search for any symbols containing "root" and print the result. Running the program again gives us this output.

Uncomment the rest of the code (be careful not to uncomment the actual comments!) which shows symbols printed using the following four methods:

• Accessing the dictionary value using the key

• Copying/pasting symbols from the result of using the search function

• Using the decimal code and the chr function

• Using the hexadecimal code prefixed with \u

I can’t see any reason to use either decimal or hexadecimal in Python, but would suggest the ideal usage would be to copy/paste any well known symbols into code, and use the dictionary key for more obscure ones. The latter option makes code self-documenting although admittedly rather long. Ultimately it’s a personal choice based on who is likely to read or maintain your code.

After you have uncommented the last lines of code run the program one more time.

As I mentioned above, if you know of any Unicode characters of a generally mathematical nature which I have missed please let me know.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

The numbers in the graphic below form the first five rows of Pascal's Triangle, which in this post I will implement in Python. The first row consists of a single number 1. In subsequent rows, each of which is has one more number than the previous, values are calculated by adding the two numbers above left and above right. For the first and last values in each row we just take the single value above, therefore these are always 1.

At first glance you might think Pascal’s Triangle is of little interest or use other than perhaps teaching very young children to add up. Nothing could be further from the truth - as with so many areas of mathematics this simple idea has spawned a large area of study and overlaps with even more. The Wikipedia article on the subject is fairly comprehensive but I am writing this article as a precursor to another on the subject of a contraption called a Galton Board, which takes us into the area of statistics and specifically the normal distribution. If you look at the last row of numbers you might be able to discern the first inklings of a normal distribution.

To that end, note that each number represents the number of possible paths to it from the top, assuming we travel downwards and either to the left or right. For example, the second number in the third row is 3, and there are three possible paths to it from the top:

• left, left, right

• left, right, left

• right, left, left

The Data Structure

Let’s start of by considering the kind of data structure we need to represent Pascal’s Triangle. For this purpose it might be simpler to show it left-aligned rather than centred.

Looking at the layout above it becomes obvious that what we need is a list of lists. In a Pascal’s Triangle the rows and columns are numbered from 0 just like a Python list so we don’t even have to bother about adding or subtracting 1.

Program Requirements

The purpose of this program is simply to print out Pascal’s Triangle to the number of rows which will be specified as a function argument. Calculating values doesn’t present any obvious problems as it is simply the addition of two numbers. However, calculating the indexes of the two numbers in the previous row to add is fiddly so I’ll use the following formula, where n is the row number and k is the column number, both 0-based:

The Code

This project consists of a single file called pascalstriangle.py which you can clone or download from the GitHub repository.

The main function simply calls the subsequent functions to create, populate and print the Pascal’s Triangle.

The create function creates a new list and then uses a for loop to add lists of 0s to it. These lists are created by multiplying [0] by the current row number; this is the reason we loop from 1 to rowcount + 1 rather than 0 to rowcount. Finally we return the list.

In populate we iterate the rows and columns, setting the values according to the formula above.

The print_left function uses similar nested loops to print each row, the :<-4d left-aligns the output within four character widths.

The print_centre function uses similar loops but also utilises the inset variable to pad the output at the left with spaces. This is calculated using an ugly formula (which I’m not proud of but at least it works!) and is reduced by 3 after each row.

We have now finished the code so can run it using this command in Terminal . . .

python3 pascalstriangle.py

. . . which will give you the following output.

You can of course change the number of lines to another value if you wish.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

I am currently working on an article about calculating sines and cosines using Taylor Polynomials. These make heavy use of factorials so I started thinking about ways to streamline the process.

Factorials are the product of a given positive integer and all the integers below it down to 1. The notation for factorial is an exclamation mark after the starting or highest integer as illustrated below.

This post consists of a simple project using memoization with a lookup table to pre-calculate factorials and store them for future use.

Many algorithms and other processes make heavy and repeated use of values which need to be calculated, often in a way which is expensive in terms of memory and/or time. It therefore makes sense to develop some sort of mechanism whereby the values which are or are likely to be needed frequently are calculated only once. The use of factorials in calculating trigonometric functions mentioned above is one example. In a future article I’ll extend the principle to calculating and storing sines and cosines themselves, but of course there are many other common examples.

When considering factorials the broad outline of memoization using a lookup table is simple and obvious: just use a list of integers the highest index of which is the highest number we want the factorial of. The factorial of a given number is therefore set and retrieved using the number as the array’s index.

The Code

This project consists of a small class implementing memoization of factorials up to a given maximum. Factorials are calculated in __init__ and stored in a list. There is also a method to get an individual factorial.

The code consists of the following files which you can find in the Github repository.

• factorialsmemoization.py

• fmdemo.py

This is the code for factorialmemoization.py.

The __init__ method takes as an argument the maximum number to calculate factorials of. We then create an empty list, store the maximum in self.memoized_to, and initialize a variable called prev to 1; its purpose will become clear in a moment.

We then append the factorial of 0 which is 1 to the list before entering a loop to calculate the rest. Note that this loop starts at 1 as we have already set 0. Within the loop we set the next factorial to the previous multiplied by the current number, hence setting prev to 1 to start with. We then set prev to the current factorial.

Finally we have the get method which checks whether the requested factorial is out of range, raising an exception if so. If n is valid it returns the relevant value from the list.

That’s factorialmemoization.py finished so we can move on to fmdemo.py.

After importing the module created above we enter the main function which first creates an instance of the FactorialMemoization class with n set to 8. We then enter a loop to print out factorials of 0 to 10. I have deliberately overrun the range to demonstrate the exception handling.

Run the program with the following in Terminal...

python3 fmdemo.py

...which will give you the following output.

As you can see we get the correct factorials up to 8 but then hit an exception with 9 which is beyond the calculated range.

The approach I have used here is to pre-calculate a range of values. This is reasonable in situations where we know of think we’ll need a specific range of values. In situations where we cannot know or reasonably assume a certain range a completely different methodology would be more appropriate: calculate values as they are needed and add them to a suitable data structure. When subsequent requests for values are made check whether they exist in the data structure and if so return the required value; if not calculate it and add it to the data structure for possible further use.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.

In this, the last of my trilogy on calculating the areas of quadrilaterals, I will demonstrate how to use the coordinates of the four corners.

Example Quadrilaterals

The diagrams below show the two quadrilaterals I will be using as examples. The corners or vertices are numbered 1-4, and their x,y coordinates are also shown. The quadrilaterals are identical (congruent) and only differ in their coordinates.

The Formula

This is the formula for calculating the area of a quadrilateral from the x,y coordinates of its corners. The numbers 1-4 correspond to the corner numbering in the diagrams, and the x and y refer to the horizontal and vertical coordinates respectively.

The formula requires only multiplication, addition and subtraction - basic arithmetic - but the large number of similar terms make the formula difficult to grok*. To make things easier the following diagram shows how the two sets of inner brackets are calculated.

* I am using this word in its original Robert A. Heinlein meaning!

This is the entire process:

• Multiply each of the pairs of numbers connected by green lines, and add up the products

• Repeat with the numbers connected by red lines

• Subtract the second result from the first

• Multiply by 0.5

The Code

This project consists of the following files which you can find in the GitHub repository.

• quadrilateralareascoordinates.py

• qafcdemo.py

This is quadrilateralareascoordinates.py which contains the function to implement the above formula.

As you can see from the docstring, the single argument to calculate_area is a list of lists in this format:

[[x1,y1],[x2,y2],[x3,y3],[x4,y4]]

We then carry out the calculations represented by the sets of inner brackets. To obtain individual values we use an expression in the format c[a][b] where a is the corner (starting from 0 rather than 1 of course), and b is 0 for x and 1 for y.

I don’t often use single letter variable names but if I had called the argument coordinates these two lines of code would have been ridiculously long!

Lastly we just subtract the second result from the first and multiply by 0.5.

Now let’s try the function out on the two example quadrilaterals in qafcdemo.py.

For each quadrilateral I have created a dictionary containing the coordinates in the required format and also an area set to 0. The areas are then set with calls to calculate_area, and finally the dictionary is printed.

Running the Program

Now we can run the program like this:

python3 qafcdemo.py

This is the output.

Leave a comment

For updates and random ramblings please follow me on Bluesky.

No AI was used in creating the code, text or images in this article.